Richard Boyd asserts that we are seeing the start of the Century of Simulation and Human Intelligence use

If you could manage to ignore the purveyors of even higher definition (4K) displays at the 2014 Consumer Electronics Show in Las Vegas, you could see the pattern emerge above the noise. Something new is happening. It is becoming clear that the last century was about the recorded moving image and the birth of computing; but the beginning of this century is about something else. It is about ubiquitous intelligence.

Richard Boyd. Image Credit: Richard Boyd
Richard Boyd. Image Credit: Richard Boyd

In the clamoring halls and private venues of Las Vegas last week there was abundant evidence that the revolution in sensors and artificial intelligence was in full swing. At a press conference there on Sunday before everything had really geared up, Shawn DuBravac, chief economist for the Consumer Electronics Association, talked about the “Age of Autonomy” and the “Internet of Things”. He was quoted in the LA Times as saying the number of intelligent gadgets in our lives is expanding at an accelerating rate and “they are increasingly talking directly to each other and making decisions without consulting us pesky humans.”

My perspective on this is that this is but an opening chapter in what will become the Century of Simulation and Super Human Intelligence. We are just now coming to grips with the Marshall McLuhan idea that the things we make, in turn make us. Our digital progeny are outstripping our understanding, beginning to evolve and emerge as new things we could not have even conceived of a decade ago. And it is time that we also become comfortable with the understanding that our poor human brains have not had an upgrade since the Pleistocene Epoch. The synapses in our poor dumb brains occur at around 200 Hz. Not megahertz, not giga; just plain old Hertz. Once robots develop a sense of humor they will undoubtedly snicker at our woeful inadequacies.

We are in an exponential age; an age of increasing complexity and uncertainty. Aided by advancing computer power we humans are creating systems thousands of times more complex than our comprehension of what emerges. It should not be surprising in an interconnected world of such complexity to see events frequently spin out of our control and overwhelm us; to see “Black Swans” and unanticipated events appear more frequently.  Grains of sand cause avalanches. Butterflies flap their wings and unleash hurricanes on distant shores.  Financial markets crash and digital viruses wrack the Internet while biological viruses take wing with air travel to sweep through entire populations. On the network-centric battlefield and in the technology-laden hospital we are asking humans to adapt to enormous complexity and perform flawlessly where mistakes can lead to death.  As complexity increases (and it shows no signs of abating) it becomes vitally important that we look for help from the machines who (which) are obviously becoming so much better than us at certain things.

In his 2011 commencement address at Harvard Medical School, Atul Gawande, MD, explained a central problem of medical practice today. He said that healthcare providers must learn a complex cooperative choreography akin to what pit crews perform during a Formula One race. He described how modern medical practice was designed during a time when there were very few medical interventions available to a practicing physician, and it was therefore conceivable for a single physician to hold himself forth as a master of all knowledge in the profession.  He goes on to say that " Resistance ... surfaces because medicine is not structured for group work" Yet, the increasing complexity of the 4,000 and growing surgical procedures and more than 6,000 drugs that an MD is legally allowed to inflict on a patient requires a cooperative team effort to avoid errors.

Technologies exist now to simulate all of the physical processes, team work flow and equipment issuing virtual models of hospitals. Image Credit: Z-Shift
Technologies exist now to simulate all of the physical processes, team work flow and equipment issuing virtual models of hospitals. Image Credit: Z-Shift

In 1975, a hospital patient required the care of 2.5 full time equivalents (FTEs). Because of increasing complexity and advancing technology,  a typical patient in 2014 requires as many as 15 full time equivalents for care.

Gawande went on to state the thesis of a number of other articles over the last two years. The engineering processes and technologies used in aviation, engineering and construction, virtually any industry that combines complexity with high risk, should be adopted by healthcare. While healthcare struggles to adopt checklists and basic systems engineering, Aviation, oil and gas and other industries are adopting a new level of team integration: the integration of teams of humans in cooperation and symbiosis with automation.

 “If being in a hospital bed made a difference, it was mostly the difference produced by warmth, shelter, and food, and attentive, friendly care, and the matchless skill of the nurses in providing these things. Whether you survived or not depended on the natural history of the disease itself. Medicine made little or no difference.”

Lewis Thomas  in “The Youngest Science”

Last year over seven hundred million passengers boarded over ten million commercial flights in the United States. The logistics and maintenance systems and engineering required to manage all of those flights safely required command of some of the most complex systems ever devised. Yet, last year there was only one (Asiana, SFO) crash with fatalities (not a US Carrier).  In fact, three of the last four years we had no fatalities in commercial aviation in the United States.  It is not because aviation is not subject to the vagaries of chaotic natural systems and human error; quite the contrary. Human flight remains an unnatural act that continues to rank highly among our greatest fears. Perhaps that is why errors are less tolerated in this domain than in automobile operation or other engineering domains. In aviation we have learned to use checklists, safety interlocks, prognostic maintenance, system of systems engineering and simulation training to essentially engineer human error out of the system.

In contrast, last year, approximately 39 million people checked into just over 5,000 hospitals in the United States. Depending on which study you believe and how you slice the statistics, somewhere between 90,000 and 200,000 of those people died unnecessarily, because of a medical mistake or error.  Yet these deaths do not inspire a public outcry and seem to go virtually unnoticed.

A recent study of routine hospital visits to 10 hospitals in North and South Carolina determined that patients have an 18% chance of being harmed by medical care. Since a 1999 report released by the US Institute of Medicine determined that medical errors in hospitals were causing close to 100,000 deaths and 1 million injuries each year there has been increasing scrutiny of patient safety measures, but those efforts are trailing woefully behind advances in technology and complexity and there has been no measurable improvement. If there was an 18% chance of injury every time someone got into a car, how many people would drive? Yet, this error rate seems to be tolerated in healthcare. What is most distressing is that technologies and engineering approaches exist to dramatically diminish these errors, but they are not being used. Only recently, under the chiding pen of Atul Gawande, have surgeons started to use simple checklists to avoid mistakes in operating rooms.

When I speak at conferences on this topic I often bring up highway safety as a good example for why we tolerate errors in healthcare. We certainly seem to tolerate errors with automobiles. As with healthcare, we have engineering solutions today that make the 35,000 annual deaths and more than one million injuries on our nation’s highways unnecessary. But we have not yet decided, as a society, that the deaths and injuries are unacceptable. Google robotic cars have safely navigated over 140,000 miles without incident. Numerous efforts have proven that robotic cars are a very viable option and the savings and safety improvement could revolutionize our economy and society; but switching from our existing highway system to a fully robotic system will require massive investment in retooling cars and adding sensors to highways. And that is the easy part. The biggest change will be asking people to give up the freedom of driving and trust in technologies that will reduce costs and increase safety. For the moment, we would rather continue to endure the deaths and injuries, the lost productivity (estimated at about 8% of GDP) from commuting in traffic and the extra energy consumed by inefficiency.

If we are not moved to action as a society by the toll in human death and suffering from these avoidable mistakes, perhaps focusing on the cost will rally us to the cause. One of the fascinating outcomes from the recent studies of waste and errors in our healthcare system is that the size of the bogey is just about the same as our annual deficit. In other words, if we could just solve this one problem, we stand a good chance of eliminating the sucking chest wound that is our annual deficit.

In the United States we spent around $2.4 trillion last year on healthcare; around 16% of our gross domestic product. Studies show that around half of this cost is avoidable error and waste, or around $1.2 trillion. While I was at Lockheed Martin, we became interested in the challenge this waste presented to our national security and began looking at healthcare and identifying areas where we believed we could help. As we went across the country meeting with leading teaching hospitals and surgeons like Richard Satava, MD at the University of Washington and Fabrizio Michelassi, MD at New York Presbyterian; and advocates for device interoperability like Julian Goldman, MD at CIMIT in Boston we found a number of areas where technologies and processes from aviation and space systems could address the healthcare crisis.

The SuperHuman Prescription

We have a timely opportunity to reduce waste and cost and improve outcomes in the healthcare system by recognizing the limits of the old system and our human capacities and gaining a comfortable fluency with automation and sensors.

Increasing intelligence:

Given the human limitations above and the trend of ever-increasing complexity, it seems obvious that we should be augmenting our healthcare providers with always-on ubiquitous intelligence.

IBM’s Watson super computer made headlines when it beat two Jeopardy champions in 2012. Since then, Watson has “gone to medical school” and is now beginning to serve as a decision support tool to help human doctors correctly diagnose disease and prescribe personalized treatment. There are some who are already warning of this Jetson-esque idea of a synthetic doctor dispensing healthcare and rendering human intervention obsolete, as many saw on TV in the early 1960s. Despite the Ray Kurzweils of the world who warn of the Singularity – a turning point where machines excess human intelligence and replace, or have no further need of, humans, it is far more likely and advantageous for us to embrace these creations of ours to become what Hans Moravec calls “ourselves in more potent form”.  If we are to conquer complexity, we must achieve fluency with our automation and no patient should accept anything less than SuperHuman healthcare.

Thankfully, access to heavy breathing super computers like Watson is not a necessity in order to incrementally improve human performance in the correct diagnosis and prevention of disease. For all of its short comings over the last few decades, the field of Artificial Intelligence has made great strides and is commoditizing rapidly along with everything else driven by Moore’s Law. Good AI systems like MASA Life out of Paris, France and Discovery Machine out of Pennsylvania and Therasim in Durham, North Carolina, have powerful and accessible systems for capturing knowledge in expert systems that can assist human healthcare providers in an expanding range of capacities.

“…no patient should accept anything less than SuperHuman healthcare.”

Since Microsoft Kinect unleashed a commodity interest in 3-D sensors for machine monitoring of human activity, a gold rush of innovation has erupted in the field of cheap ubiquitous sensors that can monitor humans and help us overcome bad habits or notify care givers or loved ones when patterns emerge that indicate illness or need of assistance.

(See Carecam as one example

Remember that the best chess player in the world is not a computer, but a team of humans working with computers and perfected algorithms. The 21st century imperative is determining how to achieve the right balance between humans and automation to optimize outcomes. Those who do master this balance and achieve a comfortable fluency with simulation, sensors and analytics will not only outperform those who do not, but they will begin to appear super human.

The 21st century imperative is determining how to achieve the right balance between humans and automation to optimize outcomes.

Super human healthcare provided by teams of humans in cooperation with increasingly powerful artificial intelligence, sensors and automation may be our only hope for staunching the sucking chest wound of escalating errors and costs in healthcare.  Thankfully, the models for this cooperation and comfortable fluency exist and are now available to be adopted by healthcare.

About the Author

With 22 years of experience working with computer gaming technologies for entertainment, design and the film industry, leading the virtual world labs group within  Lockheed Martin, and now leading a company focused on artificial intelligence and neural nets,  Richard Boyd brings a unique perspective to the looming healthcare crisis. His efforts seek to harness unprecedented, even revolutionary, recent progress in computer gaming technologies, sensors, artificial intelligence and development processes that make this an opportune time to bring the best technologies both computer gaming and aviation have to offer to address our healthcare crisis.


Ericsson, Anders K.; Prietula, Michael J.; Cokely, Edward T. (2007). "The Making of an Expert". Harvard Business Review (July–August 2007).

a study of 10 North Carolina hospitals published in The New England Journal of Medicine in November (2010;363:2124-2134)