These are Peter’s picks and there is no doubt that the implications for the planet and humanity from these advances will be significant in this decade and beyond.
Did he miss any field and scientific and technology breakthroughs that should have been included? I will consider that and maybe add some of my own in the coming weeks. So please continue to visit this site and let me know if you agree with his or my additions to his best breakthroughs of 2022 list.
The #12 breakthrough is the approval of laboratory-grown meat by America’s Food and Drug Administration (FDA). In November, California-based Upside Foods received FDA approval for its lab-grown chicken. Before the company can begin selling its product it still needs approvals from the Department of Agriculture Food Safety and Inspection Service (USDA-FSIS) which is expected soon. This marks the second country to approve lab-grown meat. The Singapore Food Agency (SFA) granted Eat Just Inc. approval for its cultured chicken in December 2020.
Of the 80 billion animals raised for meat annually, 70 billion are chickens. Demand has increased six-fold in the last century. I think the chickens of the world will thank these cultured meat companies in the years to come. And so will the environment as agricultural practices change to reflect the move away from livestock. Animal welfare advocates will also be much happier. And cultured meats may solve food insecurity issues for those areas of the planet facing agricultural challenges from climate change.
The #13 breakthrough is perennial rice. China’s Yunnan University has created PR23 rice, a perennial. They have been published in the journal Nature Sustainability. Rice is an annual which means it needs to be planted every season. But PR23 can be harvested and automatically regrows. In tests, 8 consecutive harvests were produced over 4 concurrent years. Using PR23 reduces farm labour by 60%, and production costs by 50%. Farmers in Southern China have planted more than 15,000 hectares in the last year representing a 4-fold increase from 2020. Considering that rice is a staple and food source for 4 billion on the planet, the development of a perennial version can alleviate food insecurity in South Asia, Africa and elsewhere. After this, the next food crops on the bucket list to develop perennial versions are wheat and corn.
The #14 breakthrough is the Tesla Humanoid Robot called Optimus which stands 1.7 metres (5 feet 8 inches) tall. It lifts 68 kilograms (150 pounds) and carries up to 20 kgs (45 lbs.). It travels at speeds of 3 kilometres (5 miles) per hour. Future versions will climb stairs and hills, use tools such as drills and screwdrivers, and develop other abilities.
Tesla plans to mass-produce Optimus selling each for less than $20,000. This could make robots a future household and work fixture. Elon Musk, CEO of Tesla and SpaceX sees Optimus as a part of the “future where there is no poverty, where you could have whatever you want in terms of products and services. It is a fundamental transformation of civilization as we know it.” That may be an overstatement, which Musk is known to express, but we will see.
The #15 breakthrough is the XPrize ANA Avatar called NimbRo. NimbRo won for its University of Bonn team, the $10 million prize. What avatars make possible is for humans to work through a robot in real-time without being at a location. NimbRo allows its operator to hear, see, and interact with a remote environment. In winning it teamed up with an operator to complete 10 tasks earning a perfect score. Avatars make it possible for humans to do tasks in areas that normally would be hostile to them: think disaster relief, nuclear reactor accidents, dismantling a bomb, or search and rescue operations.
The #16 breakthrough is robotaxis getting approvals for use in public transportation. Alphabet’s Waymo and GM’s Cruise are two leading companies developing these autonomous vehicles. Waymo was the first to offer fully autonomous public rides which began in Phoenix, Arizona in 2020. In November 2022, it won approval from the California Public Utilities Commission to carry passengers in robotaxis without a safety driver present. GM Cruise has operated a robotaxi service in Austin and Phoenix and recently was approved to offer a fully driverless ride-hailing service in San Francisco.
The use of robotaxis will lower the cost of personalized transport and should increase car-sharing taking more cars off the roads and streets of cities. This ultimately can lead to less land used to support automobiles. That means fewer parking spaces, more manageable road networks, and turning asphalt and concrete into parks and sustainable housing. This should help reduce the urban heat island effect in cities and mitigate global warming.
The #17 breakthrough is eVTOLs which stands for Electric Vertical Takeoff and Landing vehicles. The emergence of flying cars made the U.S. Federal Aviation Administration propose new rules in 2022 to help pave the way for commercial air taxis to operate by 2025. The goal is to have them in Los Angeles in 2028 for the Olympic Games.
Top companies include Joby Aviation which this year received Air Carrier certification and formed a partnership with Delta Air Lines. Another company, Lilium, demonstrated the Phoenix-2, an eVTOL that transitions from hover to wing-borne flight. Saudia Airlines signed an MOU to purchase 100 and develop an eVTOL network across Saudi Arabia. Archer Aviation formed a partnership with United Airlines to deliver 100 eVTOLs for passenger service between Newark Airport and downtown Manhattan. And BETA Technologies demonstrated human-piloted eVTOLs winning military airworthiness approval.
It looks like eVTOLs are about to take off in major metropolitan cities with the market estimate for these services to be worth $30 billion by 2030. As eVTOL networks grow this will change where and how we travel making difficult-to-reach geographies more accessible.
The #18 breakthrough is in the proliferation of Qubits in new quantum computing platforms. Qubits are the equivalent of the bits and bytes that can be found in today’s computers. Several significant advances in 2022 have included extending qubit lifetimes by a factor of 5. Other advances include tuning large numbers of qubits to interact with each other while maintaining coherence using what is called quantum many-body scarring (QMBS). These advances address the biggest challenges in quantum information science. Longer coherence times mean more useful qubits for applications in physics, chemistry, medicine, transportation, finance and other fields. McKinsey estimates that four industries, in particular, automotive, chemicals, finance, and pharmaceutical will see quantum applications worth $700 billion by 2035.
The #19 breakthrough comes with IBM unveiling its largest quantum computer yet called Osprey, a 433-qubit machine that is three times larger than the 127-qubit Eagle which the company announced just last year. The new processor has the potential to perform complex quantum calculations far beyond the capabilities of any classical computer. As IBM points out, “the number of classical bits that would be necessary to represent a state on the IBM Osprey processor far exceeds the total number of atoms in the known universe.” And IBM has plans for a 1,000-qubit chip to be released in 2023.
The larger the quantum computer, the greater its ability to tackle more complex problems. For example, Toronto-based Xanadu this year used a 216-qubit quantum chip called Borealis to tackle the “Gaussian boson sampling” problem, a benchmark used to measure quantum computing prowess. The Borealis did it in 36 microseconds. A conventional computer would have taken over 9,000 years.
The #20 breakthrough is progress using synthetic data to accelerate artificial intelligence (AI) systems development. Data feeds AI systems. Good data is great. Bad data, not so much. The real world can be messy and data can be riddled with biases. Gartner estimates up to 85% of AI projects end up delivering erroneous outcomes because of data bias. That’s where synthetic data comes in.
Synthetic data is artificial. It mimics real-world observations. It is used to train AI machine-learning models. This year researchers at MIT, IBM Watson, and Boston University built synthetic datasets from 150,000 video clips capturing a range of human actions. Machine-learning models used the clips for training. Videos with fewer background objects produced better outcomes than AI using real data. What this potentially means is that synthetic data by 2030 will overshadow real data in AI models and will save hundreds of millions of dollars annually.
The #21 breakthrough is text-to-image generation. Open AI’s DALL-E 2 and Stability AI’s Stable Diffusion are advanced machine-learning text-to-image platforms. DALL-E 2 is built to understand natural language and turn it into images, creating creepily accurate renderings. Today, 1.5 million active users are working with it to create over 2 million images daily at no cost. OpenAI has partnerships with Microsoft, Shutterstock and others to increase the platform’s use and influence. Stable Diffusion’s Stability AI is currently used by more than 10 million people. And apps from TikTok to Lightricks are also AI-powered art generators.
For creatives doing art and design, these tools are game changers. A recent New York Times interview with several creative professionals talked about how partnering with AI was allowing them to generate more creative ideas faster. Filmmakers are using the tools to create pitches for studios. And generative AI isn’t limited to images. It can also be used to generate computer code from text. That can make anyone a programmer.
The #22 breakthrough and the last in this list is ChatGPT, a conversational chatbot created using OpenAI and the recently released GPT 3.5 language. This chatbot is a tool for AI-human collaboration and works by taking conversational input and producing code or text output. It can be used to answer coding queries, run virtual computing platforms, write song lyrics, and even novels. When OpenAI released ChatGPT on November 30, 2022, by December 5th over 1 million people were using it. What will be the new forms of art and business that this type of AI tool creates?
Peter Diamandis concludes his 22 breakthroughs with this comment related to the implications of advances in AI, stating: “By the end of this decade, there are going to be two kinds of companies, those that are fully utilizing AI and those that are out of business.”
]]>LIDAR wouldn’t exist without lasers. Initially developed for military applications, today LIDAR is used widely to measure and scan in numerous fields from agriculture to transportation.
Increasingly the focus on the use of LIDAR has been in the automotive field to provide vehicle assistance to drivers, and in the future to make vehicles driverless.
Autonomous vehicles are being pursued by all of the major automobile makers. No one has done more to promote the idea of autonomy than Tesla. But the company has been reluctant to adopt LIDAR on all its models. Only the Model Y features a limited LIDAR set. Elon Musk, the company’s CEO, has called LIDAR a crutch.
Yet LIDAR is the most likely tool to get Tesla and every other automobile manufacturer to the ultimate goal of a fully autonomous road experience. Tesla, instead, is relying on cameras, ultrasonic systems, radar and a raft of software and analytics to assist with navigation. This pursuit of autonomy without LIDAR is showing its weaknesses and delaying the company’s driverless vehicle goal. Too many crashes of Tesla models have been cited in the media.
The limitations of these other imaging technologies are well known. Cameras can take great moving pictures but provide poor depth perception. Cameras don’t like it when the Sun blinds them.
Ultrasonics are only good to do short-range detection.
Radar is better than ultrasonics for detecting what lies ahead but not as good as cameras in providing images. Radar sees what cameras do not see when the Sun gets in the way, or if there is fog or other inclement weather conditions that make visual navigation difficult. But radar images don’t provide detail.
LIDAR, however, delivers where these other imaging technologies fail.
LIDAR-assisted steering maps what lies ahead in three dimensions. It creates precision real-time visualizations of the surrounding environment. LIDAR can see objects in the immediate path and map what is happening on the road well ahead.
LIDAR technology implementation can be light or heavy. The three vehicles seen below show where and how LIDAR-equipped sensors can be implemented in light (L1), intermediate (L2) and heavy (L3) modes. The vehicle on the left can tell a human driver information to help he or she navigate the roads and obstacles ahead. The vehicle on the right, however, can make the human driver redundant.
Will we see fully autonomous vehicles become predominant on streets and highways by the end of the decade? With the cost of the technology continuing to drop, and the explosion of sensors that integrate LIDAR, it certainly appears that is where we are heading.
ADAS is an acronym for Advanced Driver Assistance Systems. LIDAR sends out hundreds of thousands of laser pulses per second. lt uses navigational systems like GPS, and photodetectors to provide a real-time ongoing picture of what is in front, at the sides, and behind a vehicle. The information it continuously ADAS continuously receives allows it to make precise calculations and decisions as the vehicle is moving. Unlike radar and cameras, LIDAR has the ability to see what lies not just nearby but well ahead. That’s why automobile manufacturers see it as critical to ADAS evolving into fully autonomous vehicles.
It should be no surprise that in a recent Bloomberg survey, seventeen automobile manufacturers indicated they were heavily invested in LIDAR companies and systems for use in many current and future models. Even Tesla is coming around to the realization that LIDAR is an essential component to creating vehicles safe enough to drive themselves.
]]>
At the most recent Consumer Electronics Show (CES) in Las Vegas in January, a new wave of autonomous vehicles (AVs) had their coming-out party. Many of these are concepts and some will survive the cut and be seen on future roads. Much of the focus remains on the technology within these vehicles. But technology embedded in road infrastructure will make AVs work and revolutionize urban transportation.
How is this coming about? Through advancements in AI and 5G network connectivity, the technology for smart-road infrastructure is offering the promise of roads, bridges, and other types of transit systems capable of doing real-time traffic analytics and tackling the most challenging road safety and traffic management problems.
In this article, we look at technology at the center of this transportation revolution: AI-enhanced cameras and LiDARs technology.
Today there are hundreds of thousands of traffic cameras deployed in the U.S. alone, and even millions more when CCTV cameras are considered. These are mainly used for road monitoring and basic traffic management applications. Bringing the latest advancements of AI to these assets, however, immediately improves their basic application performance and unlocks more advanced software applications and use-cases.
AI and Machine Learning deliver superior sensing performance over traditional computer vision in legacy cameras. The two enable more robust, flexible, and accurate detection, tracking and classification of all road users with algorithms that automatically adapt to various lighting and weather conditions. In addition, they provide predictive capabilities to better model road user movements and behaviours and improve road safety. This creates immediate benefits in applications such as road conflict detection and analysis, pedestrian crossing prediction (see image at the top of this article) and infrastructure sensing for AV deployments.
LiDARs can provide complementary and sometimes overlapping value to cameras. But there are several cases where LiDARs technology does not perform well, for example, heavy rain and snow. In addition, today’s LiDARs remains expensive to deploy at scale because of high unit price and limited field of view. For example, it takes multiple LiDARs for deployment per intersection where one 360-degree AI camera would prove to be a more cost-effective solution. For many budget-focused communities, therefore, AI cameras remain the technology of choice. Over time, however, as the cost of LiDARs technology drops, transportation networks may want to add it to augment the infrastructure.
As the cost of LiDARs drops, it will be viewed as a strong addition to AI-enhanced cameras being installed today. Similar to autonomous vehicles, this fusion of different types of sensors will become the go-to approach for smart infrastructure solutions. A comparison of the two technologies follows:
Feature |
Legacy Camera |
AI-powered Camera (1) |
LiDARs |
AI-powered Camera and LiDARs Fusion |
Challenging lighting (low light, glare) |
Low |
Medium |
High |
High |
Adverse weather conditions (snow, rain, fog) |
Low |
High |
Medium |
High |
Localization |
Low |
Medium |
High |
High |
Classification |
Low |
High |
Medium |
High |
Affordability |
High |
Medium |
Low(2) |
Low(2) |
Today cost-effective and performing AI-powered cameras are helping municipalities with traffic management. The addition of LiDAR in the near future, however, will, in the end, significantly improve community road safety, overall traffic flow, and diminish vehicle crashes and fatalities.
]]>California has a reputation for embracing new technologies, especially evident in its policy towards self-driving vehicles. One of those technologies that has received approval is Silicon Valley-based Zoox, founded in 2014. The state has granted the company not only the right to test its technology on public roads, but also to transport passengers in its vehicles. This is a first for California.
Zoox is building a 100% autonomous vehicle transportation service with a focus on passengers, not drivers. Their vehicles have no “front” or “back.” There is no steering wheel. Built specifically for safety, Zoox is focused on eliminating the human factor in driving and therefore, accidents.
With the California approval, Zoox is now expanding its operations from allowing only its employees and family members on its vehicles, to the general public. The company wants to hear from Californian commuters so that it can improve its self-driving technology.
Zoox is committed to sustainability, and therefore, all of its vehicles are zero emission.
Before the company can achieve an absolutely autonomous vehicle service, California is requiring that at least one trained driver be onboard just in case of an emergency. The agreement also stipulates that the current operation of Zoox vehicles will be free to anyone who chooses to go for a ride in one of its vehicles. These restrictions fall in line with California’s Drivered AV Passenger Service pilot program, which lawmakers approved in May of 2018. Officially listed as Decision 18-05-043, the main requirement stipulates a physical driver on board at all times.
Along with this pilot, California has also issued a second called the Driverless AV Passenger Service program which could allow companies like Zoox to send their vehicles out without a physical driver on board. Under this California pilot, the DMV requires the company to have employees constantly monitor their vehicles from secured locations. In addition, every vehicle must include a way for passengers to get in touch with the employees who are tracking the vehicle. Although no Zoox vehicles have been given the green light to operate this second pilot, it is likely that Zoox will be the first to be granted this privilege, all things going well.
Zoox is just one of 62 autonomous vehicle companies operating in California. But one, in particular, stands out as a prime competitor. That is Waymo, a subsidiary of Alphabet that started in 2009. Waymo has been testing its autonomous vehicles without drivers in many California cities, but to-date they are not allowed to pick up passengers.
Waymo, however, has been operating with greater freedom in Arizona. Just recently, it announced that customers in Phoenix could pay a fare to use its self-driving vehicles. While many Waymo vehicles have a human driver on board, a few now are driverless. Testing has been ongoing for several months.
When discussing the pros and cons of autonomous vehicles, critics most often question safety standards. It’s no wonder this is an issue since the press seems to cover any autonomous vehicle crash turning these rare occurrences into global headlines. The latest occurred in Tempe, Arizona, when an Uber’s self-driving vehicle fatally struck a pedestrian during a test-run. After this tragic accident, Arizona’s Governor ordered Uber to stop testing its self-driving vehicles on public streets.
As of today, there is no clear evidence to indicate that autonomous vehicles are safer than those operated by human drivers. The sample is far too small. Only putting more autonomous vehicles on the road both with and without a human driver will produce enough data to better understand the technology’s safety issues.
In the meantime, many who are technology savvy have suggested research into engineering ethics to address public fears. Through the specifying of autonomous vehicle behavior acceptable to the public, it will help companies like Zoox and Waymo to improve on their self-driving algorithms.
Safety concerns will always be the most significant issue surrounding autonomous vehicle technology. But there are other challenges. For instance, vehicle accident attorneys are grappling with changing the laws governing accidents to account for this technology. With different degrees of autonomy in most self-driving vehicles, lawyers are trying to figure out how much liability should be assigned to drivers or manufacturers in the event of a crash.
There’s also a concern that autonomous vehicles cause significant changes to the automotive industry. The growth of the sharing economy, some economists believe, will make autonomous vehicles more attractive to ride-share companies than to private consumers. To increase safety and reduce both traffic and emissions, it’s more likely that self-driving vehicles will be operated as fleets owned by the manufacturers, or by transportation businesses rather than be purchased by individuals. If this comes to pass then car dealerships and the service industry that is currently the business model around the world will drastically change. That means dramatic disruptions to post-vehicle manufacturing jobs. On the flipside, a shift to shared autonomous vehicles will mean more people will be spending time looking at screens while in transit, and that may prove a boon to advertisers.
To address some of these issues, CA DMV has been formulating autonomous driving regulations since the beginning of 2013. Currently, there are three different tests to which autonomous vehicle companies can apply: testing permits with a driver, driverless testing permits, and public use permits. You can learn more about these at the California DMV website. The full text of the California autonomous vehicle regulations is available at California Vehicle Code Section 38750. And for a list of current autonomous vehicle permit-holders, check out the official webpage.
To make inroads into the AV realm Ford purchased a majority stake in an autonomous vehicle artificial intelligence (AI) company, Argo AI, headquartered in Pittsburgh. ArgoAI has put its technology in vehicles doing pizza delivery for Domino’s, The company is testing its technology by partnering with Domino’s, parcel delivery for PostMates, and most recently has inked a service delivery contract with Walmart.
Currently, all test vehicles have human drivers backing up the AI systems. But when Ford launches its service the drivers will no longer be necessary. Nor will the cars have steering wheels, gas, or brake pedals. Ford’s AV won’t be a modified version of one of its existing models such as the one depicted in the picture below. Instead, this will be an entirely new model incorporating AV technology into every aspect of the design. The plan is to configure models for passenger, cargo, food delivery, and other services. Besides the onboard software, the AV will include a sensor array with lidar, radar, and multiple cameras.
As Ford imagines the uses for its AV it sees consumers eventually calling up a vehicle through a mobile app. going to places, using it to order and deliver groceries, having it pickup restaurant orders, and anything else that customers might find a useful service. Ford wants to learn from its customers what they would like an AV service to provide and, in time, anticipate their needs.
In the latest demonstration of its technology, Ford took reporters on a trip through the crowded streets of Miami. One passenger described the journey as the most complex scenario he had ever experienced in an autonomous vehicle.
Andrew Hawkins, writing for the Verge, states, “I’ve ridden in about a dozen autonomous vehicles in a variety of settings over the past three years, and this was one of the most complex….I’ve ever seen an AV navigate. I’ve been an AV passenger in San Francisco, Mountain View, Las Vegas, Detroit, and Pittsburgh, but nothing like Miami — cars and trucks and candy-colored mopeds jockeying for space with bicycles, scooters, and old ladies with laundry-filled pushcarts. Despite the heat, it’s the type of environment that would make most AV operators break out in a cold sweat.”
Hawkins describes the ride as “a marvel,” with the vehicle navigating construction zones, left-hand turns, lane markers, intersections, and pedestrians. The only time the human driver had to take over happened when a passing truck caused a road dust cloud. The human driver pointed out that there was no attempt to game the system and that the demonstration was authentic.
Ford’s spokesperson at the demonstration indicated that of the 2,000 incorporated features to its autonomous vehicle technology, the company has already perfected 1,800. The 200 remaining include road dust clouds. You can imagine that falling snow, as well as snow on the ground and roads, could be among the remaining items. Ford has worked on solving snow since 2016 and is developing what it calls vehicle“snowtonomy.”
Ford has committed $4 billion through 2023 for its autonomous vehicle service and has turned a vacant Detroit century-old office tower into a new headquarters for electric and AI-driven cars. The company estimates the market for AVs to grow to $330 billion by 2026.
]]>
A Haifa-based company, Tactile Mobility, formerly known as Mobi-Wize, intends to make future autonomous vehicles tactile and sensory smart. It launched in an announcement today, its tactile sensing and data analytics software along with agreements for five proof-of-concept trials planned with North American and European automobile manufacturers. Amit Nisenbaum, who recently has joined the company as CEO, describes Tacticle Mobility’s value proposition:
“When you drive you can feel the tires as they slip on ice, or sense the engine is having trouble despite it not registering with your car’s vision-based electronic systems…This missing tactile sense is a major hurdle in the development of advanced and autonomous vehicles and a significant limitation when it comes to mapping road condition data that has been ignored until now.”
This represents an important technology enhancement in the five-stage process that marks the evolution of vehicle autonomy. If you are unfamiliar with what these stages are here is a quick summary:
Stage 1: Autonomy features such as adaptive cruise control (My RAV has this).
Stage 2: Automatic braking and correcting lane drift as well as fully autonomous parking. (My RAV has the first two but not the third).
Stage 3: Autonomy where the vehicle handles all driving functions related to navigating on a highway. At any point, a person in the driver seat can take control.
Stage 4: Autonomy that allows vehicles to navigate in a complex urban setting without driver assistance, although a person in the driver seat can take control when needed.
Stage 5: Autonomy where no driver seat exists as well as no steering wheel, accelerator, or brake pedal. All who enter the vehicle are passengers.
Tactile Mobility’s technology is of value at all five stages collecting crucial, real-time non-visual sensory data that tracks wheel speed, wheel angle, engine RPMs, automatic transmission paddle position, gear position, and more, analyzing all this information to yield actionable insights. Tactile Mobility’s software feels the road, senses the drive as it is affected by vehicle weight distribution, the quality of the road surface, and by the grip of the tires.
The software is vehicle agnostic, that is, it can be implemented with almost any manufacturers’ brand. I suspect we will be hearing about tactile data intelligence in future models of vehicles as these companies accelerate their plans to bring full autonomy to the world’s roads.
]]>
Driverless cars are coming soon — but not that soon because developers still need to work out some of the kinks in the technology. Autonomous vehicles have problems. They struggle to cope with construction work, potholes, unmarked country roads, snow, and icy road surfaces just to name a few challenges. [If you read yesterday’s posting on autonomous vehicles and ground penetrating radar, most of what Roberson describes should be addressed by a $100 US technology produced by WaveSense.]
A bigger outstanding concern is that autonomous vehicles are controlled by computers, and we all know that where there are computers there are hackers. A hacker could turn a car ride into a nightmare whether you are using an autonomous car service from Uber or Lyft, or your own vehicle. One minute you’re gliding down the road, listening to a podcast, texting friends or dozing after a long day at work, and in the next moment, your car destination changes. Your doors and windows lock. A message from the hacker appears on the LED display to inform you are being held hostage. An even more frightening scenario is one depicted in spy novels. Your autonomous ride starts careening over the median on the highway or stalls out on a train track.
Developers of self-driving vehicles are aware of these threats and are working to make sure riders never get placed in these situations. Separating reality from fiction means defining the threat in real terms.
The threat hackers pose is certainly real. Just ask Wired journalist, Andy Greenberg, who in 2015 took part in an experiment in which the car controls of a Jeep he was driving were taken over by hackers. Security researchers Charlie Miller and Chris Valasek were able to control the Jeep’s steering, brakes, keyboard functions and transmission remotely through the Internet. They could kill the vehicle’s engine and send it off the road with Greenberg sitting behind the wheel.
The demonstration caused a massive recall of more than one million of Fiat Chrysler vehicles, as the manufacturer scrambled to patch security issues with its vehicle software. And that happened in a driver-controlled car where a human behind the steering wheel could intervene by pushing on the brakes or trying to regain directional control. Now imagine the same happening in an autonomous vehicle. Things related to security get far more complicated.
If you are a recent buyer of a new vehicle the chances are it is all ready packed with semi-autonomous features. It may sense obstacles and brake automatically if you don’t press the brake pedal yourself. Some cars even park themselves. With a steering wheel and brakes, if something goes wrong with these features, you can still gain control. But in an autonomous vehicle without a brake pedal or steering wheel, you can’t do that. You might not even be in the front seat. Without manual controls, you are very much at the mercy of the vehicle’s autonomous computer controls.
Vulnerability to hacks is not limited to controlling brakes and steering. Increasingly, vehicles come with high-tech bells and whistles like Bluetooth or digital touch-screen displays that are equally vulnerable. Bluetooth, onboard Wi-Fi, OnStar cellular radio and dashboard display web browsers represent points of entry where hackers can take control.
As the auto industry moves away from the model of individual ownership and closer to a car sharing alternative where users order rides as needed, Miller and Vasalek predict vehicles will need to pare down fancy digital features for security reasons.
The driverless cars of the future may not have radios, Bluetooth, or touch-screen displays hard-wired into the dashboard. Riders probably won’t miss them. Most will use personal devices for listening, and reading. The need for hands-free calling will go the way of the Dodo when passengers no longer have to control a steering wheel. Riders will use a locked-down, bare-bones tablet to interact with an autonomous vehicle with the retention of some traditional manual controls or other means of performing a manual override in case hackers try to take control remotely.
In the U.K., lawmakers are already seeking to criminalize the hacking of autonomous vehicles and are making attempts to ensure that humans remain responsible for road accidents, even if it not responsible for piloting the vehicles involved.
Ultimately driverless vehicles are expected to save millions of lives by making road travel safer. That includes safe from those who would exploit their software controls. For this reason manufacturers and developers of autonomous software are working hard to make the technology as hacker-proof as possible.
]]>
In 2016, MIT’s Lincoln Laboratory developed real-time, nighttime, centimeter-level vehicle location at highway speeds on roads where lane markers and other features were hidden by fallen snow. They did this using ground-penetrating radar to detect what traditional detection systems in existing autonomous vehicles could not. That’s because autonomous vehicle developers use optical detection for navigating roads and have been challenged by weather conditions that cover surfaces making it impossible to visually identify the lane and side-shoulder markers. Using very high frequency (VHF) radar mounted beneath vehicles made it possible to see these important points of reference in real time. Equipped vehicles were easily able to navigate in bad weather conditions at 100 kilometers per hour (60 miles per hour) highway speeds. The test results from 2016 were published in an article appearing in the Journal of Field Robotics.
The technology MIT developed is called Localizing Ground Penetrating Radar, or LGPR. Its main component is a waterproof, 12-element antenna array mounted under the vehicle and costing about $100 US per unit. The array uses both VHF and stepped-frequency continuous waves to penetrate the ground providing a detailed road and beneath-the-road map which can be compared by the onboard autonomous vehicle system with its existing maps and visual imaging.
The immediate application has been military with the LGPR device mounted on tanks and armored personnel carriers and tested in Afghanistan and Iraq to detect explosive hazards on supply routes and in open ground areas.
The next step in the evolution of LGPR may come from a new MIT spinoff startup company, WaveSense, which is taking the technology from the laboratory to commercial application in autonomous vehicles.
Other applications include utility companies mapping buried power and pipelines, and cities needing to create a subterranean map of infrastructure below the surface.
WaveSense LGPR has demonstrated an ability to see 3 meters (10 feet) below the road. For the purpose of autonomous vehicles, the penetration need only be beneath an ice or snowy surface, no more than a few centimeters directly below and laterally. It works even when cameras or other autonomous technologies like LIDAR fail. And should the LGPR fail then it is likely the optical cameras and LIDAR will remain operational, which means equipped vehicles will have multiple ways to ensure they can navigate autonomously.
]]>
How? A global network of connectivity from drones, satellites, devices, and sensors are creating an Internet of Everything. Diamandis describes the four different levels in this evolution:
As always feel free to share your comments.
This is the coming age of microsatellite constellations brought to you by SpaceX and OneWeb.
OneWeb is working on almost 900 satellites, while SpaceX will be deploying over 12,000 mini-fridge-sized ones. Both have plans to create a global 5G Internet, but that represents only a fraction of the potential of what similar deployments will bring: microsatellite equipped with high-resolution cameras will extend the Internet of Things (IoT), providing a massive amount of data to help solve the world’s grand challenges.
As of August 2017, there were nearly 1,800 operational satellites in orbit. Of these, 742 were communications satellites, 596 were used for Earth observation, and 108 were used for navigation. And a massive increase in operational satellites is on the way at they become smaller and as launch costs plummet. Today private companies all over the world are building satellite technology. Chinese companies plan to place 60 commercial high-resolution Jilin 1 imaging satellites in orbit by 2020. Planet Labs, a disruptive company that uses milk-carton sized imaging satellites to help industries obtain game-changing data, is showcasing over 175 satellites in orbit that can image anywhere on the globe with up to 3.72-meter resolution. The company is also offering a specialized, targeted satellite option called SkySats. Thirteen of these can achieve up to 72-centimeter resolution. The SkySats can also capture video, which can be used to extrapolate 3D models. These satellites are built using the same technology Google deployed to capture crisp 3D-image views for its Google Maps.
Closer to Earth’s surface, a few hundred feet above our heads, we’re developing an extensive network of autonomous drones to collect valuable information for farmers, wind turbine surveyors, financial institutions, and many others. At this year’s Consumer Electronics Show in Las Vegas, the U.S. Department of Transportation announced that over 1 million drones were registered with the Federal Aviation Administration (FAA). By 2020, the FAA predicts 7 million drones will be flying in North American skies. While private and commercial drone flight rules remain moderately restrictive, this past October, the U.S. announced plans for the federal government to work with companies to start deploying large fleets of drones with more relaxed restrictions.
During 2017 drone data collection was invaluable for saving lives in natural disasters (hurricanes and wildfires), surveying damage and providing search and rescue operations with crucial footage of hard-to-reach locations. And as drones fly more and collect more data, a small fleet will easily generate up to 100 terabytes of data per day.
Intel predicts that the self-driving car industry will grow to $7 trillion by 2050. On the ground the arrival of these vehicles will represent a data collection revolution as they image and sample their surroundings all the time. Imagine millions of autonomous vehicles on streets, each packed with dozens of cameras, LiDAR and radar to help them navigate. Waymo, one of the leaders in developing this technology, uses three different LiDAR sensors, five radar sensors, and eight optical cameras. The sensor suites on a single car generate more than 4 terabytes of data per day. Just one technology, LiDAR, a laser detection system that bounces its light beams off surrounding objects can create up to 9 million data points per second according to market leader Velodyne.
Tesla, unlike Waymo, avoids using LiDAR altogether, and instead opts for an ultrasonic, radar and camera approach to autonomous vehicles. Here’s a description from Tesla about the sensors on each of their cars:
“Eight surround cameras provide 360 degrees of visibility around the car at up to 250 meters of range. Twelve updated ultrasonic sensors complement this vision, allowing for detection of both hard and soft objects at nearly twice the distance of the prior system. A forward-facing radar with enhanced processing provides additional data about the world on a redundant wavelength that is able to see through heavy rain, fog, dust and even the car ahead.”
The bottom line is this: when we enter the era of autonomous cars, there will never be a car, pedestrian, accident, or street-side pick-pocket that isn’t being imaged. These cars will record an extraordinary number of images in detail.
And then there is augmented reality. While the world now has more mobile phones than humans with smartphone numbers alone reaching 3.5 billion by 2021, we will soon see the emergence of augmented reality (AR) headsets. AR glasses will feature a multitude of forward-looking cameras that image everything at a sub-millimeter resolution as you travel through the day. And by the end of 2020 these AR glasses, and smartphones, plus medical wearables and smart dust will amount to 50 billion connected devices hosting more than a trillion sensors.
There has been an exponential growth in the number of sensors per connected device over the last decade. Sensors on phones have doubled every four years. This means expect to see 160 per mobile device by 2027, and a world jam-packed with nearly 100 trillion in total. And almost every one of these sensors will be accessible to artificial intelligence delivered over the Internet or in specific devices.
I can envision waking up in the morning, putting on my augmented reality contact lenses, and forgetting about them for the rest of the day while they record every conversation, every person crossing the street, and everything I look at. Drawing from this constant stream of observational data, I’ll be able to train my social graph and preferences into my AI using the collected data.
The bottom line is we are heading towards a future where you can know anything you want, any time you want, anywhere you want. In this future, it’s not “what you know,” but rather “the quality of the questions you ask” that will be most important.
If you are in the fashion business and want to know the average spectral color of women’s blouses on Madison Avenue this morning, ask your AI and it will gather the image data and provide an accurate answer in seconds. Then you can ask the AI if any recent advertising campaign correlates to blouse colour choices.
Such abundance of data is what I call “Radical Transparency” and it leads to a few interesting conclusions, which will no doubt be the topic of a future posting.
In the world of IoT and 1oo trillion sensors privacy as we have known it will truly be a thing of the past.
And it will be harder and harder to do anything in secret without leaving a traceable digital record.
Baking the “rules of the road” into the brain of an artificial intelligence is not easy. That’s because so much in driving is not just about hard-fast rules. Some we break to avoid hitting a vehicle. Some we break because common sense dictates that we should. Human drivers, as a result, cannot be relied on to behave predictably.
In an article that appeared in The Globe and Mail last February 5th, Jason Tchir asks the question “will self-driving cars understand the finer points of traffic etiquette?” He talks about pedestrians stepping off the curb into the street and making eye contact with a driver to get a nod for his or her okay to cross. It is hard to imagine that same scenario playing out with an autonomous vehicle (AV). How would a driverless car engage that pedestrian? The camera technology would see the person but a non-verbal cue like a nod would not be possible. Toyota, Nissan, and Ford have wrestled with this issue in their testing of AVs.
The Toyota and Nissan solution at present is to have their cars display text on the windshield informing the pedestrian that it is okay to cross. The Nissan concept with text display can be seen below with the words “after you” projected on the windshield. But think about this as a solution. It means in every market where Nissan sells its vehicles it has to have messages that display in the local language. But what if there are many local languages. In Canada, for example, the country has two official languages, French and English. In places like India with many official languages, how can an AV know which language to present to a crossing pedestrian?
The Ford solution is to use a bright light displayed at the top of the windshield. The light could serve two purposes. First, it could let the pedestrian know that a Ford car with this type of light is an AV. That doesn’t tell the pedestrian anything about having the permission to cross. But another scenario could where the light displayed produces a series of flashing patterns, a sort of Morse Code, that would tell the pedestrian it was okay to cross. Hardly an intuitive solution, it would require the pedestrian to know the meaning of the messages created in the blinking lights.
Another possible solution is to dispense with lights and word messages, and instead use icons which could be animated to march across an AVs windshield. For example, the permission for the pedestrian to cross would come in the form of an animated projection of an image of a person crossing the street. But even icons don’t account for every type of pedestrian. Would an icon, or a message, accompanied by sound be a better solution? Not when the pedestrian making his or her way across the street is blind or deaf, or both.
Paul Godsmark, Chief Technology Officer, the Canadian Automated Vehicles Centre of Excellence, is quoted in Tchir’s article stating “I think there’s value in the car telling pedestrians what it’s going to do, as long as we come up with a national standard.” It is this idea of developing a national standard that is discussed at length in a May 12th, 2018 issue of The Economist, in an article entitled “Robotic rules of the road.” In it the writer asks if the rules of the road can be defined mathematically.
A number of companies are trying to do just that. Both Mobileye, and Voyage, two companies developing AV software have published papers on the subject. Mobileye’s is entitled “Responsibility-Sensitive Safety” while Voyage’s is entitled “Open Autonomous Safety.” In both the effort is to apply mathematical rules to vehicle activities. Both companies have proposed that these beginning efforts become the basis for an open industry standard to which other companies can contribute.
Even with an open standard will it cover the cultural nuances of what a honk means to drivers in different cultures? Or will it understand different driver hand gestures, both polite and impolite? And what about cars with bumper stickers or those waving, bobbing figurines that festoon the back windows of some vehicles? Or the stuff people hang on car mirrors? This is human behaviour that is far more difficult to mathematically quantify for an AV than a simple lane change or merge into traffic.
Undoubtedly, once an acceptable standard becomes common to all AV developers the act of driving will become far safer. By how much? In a recent study published by the RAND Corporation, concluded that if AVs were only 10% safer than human drivers, more than a half million lives would be saved over a 30-year period.
]]>