My Braze(n) Journey Inventing Smart Wheelchairs

It was the summer of 2006 and I had just joined the Intelligent Assistive Technology and Systems Lab (IATSL) at the University of Toronto run by Dr. Alex Mihailidis. I still remember a lab meeting where Drs. Geoff Fernie and Rosalie Wang were introducing one of their projects, an anti-collision wheelchair. As a lab that primarily focused on technologies for older adults, the issue they were trying to solve was that of exclusion of older adults with dementia from the use of powered mobility. I got to see this problem myself when I visited a long-term facility for the first time along with my supervisor (Dr. Alex Mihailidis, who would become a RESNA President and Fellow, and my co-founder at Braze Mobility Inc. several years later). Many residents at the facility did not have the strength to self-propel their manual wheelchairs, and powered mobility use was prohibited due to safety concerns. Statistics showed that 60-80% of residents have dementia, and there was not a lot of research evidence at the time (still isn’t!) to provide guidance on who might and might not be able to operate a powered wheelchair safely.

Several solutions had already been implemented and tested with the intended users – one of the things that attracted me to this lab, as it was often the case that assistive technologies were tested with able-bodied individuals, or just in simulation experiments with no users at all. 

One of these solutions included a bumper skirt, a bunch of panels installed around the powered wheelchair that would automatically stop the wheelchair when they made contact with an obstacle. The contact force required was small (1N from the time of first contact to the full stopping distance of the powered wheelchair). 

Another interesting solution was a haptic joystick – a joystick that used a bunch of motors to prevent the user from pushing the joystick in the direction of an obstacle, as part of a multi-modal feedback system that provided user feedback to indicate which direction had the most free space around the obstacle (through arrows and audio cues). One study employed a “Wizard of OZ” paradigm. The simplest way I describe this is “fake it till you make it”. The obstacle detection part of the problem was solved by a human, the “wizard”, who was following closely behind the user and would activate the feedback module whenever the user approached an obstacle. The “wizard” is typically hidden so that the user believes he/she is interacting with a system that actually offers all the functionality being simulated.

This paradigm is fairly popular in the field of human computer interaction, especially in the early stages of research where complex technologies can be mocked or simulated by a human to understand their impact on the intended user. This approach can save a lot of resources that would otherwise be spent on building the technology, and can instead help quickly identify usability issues even before the technology is built. Wizard of OZ would eventually play a big role in my post-doctoral research, helping me to accelerate my own research and development efforts. Fun fact: the Wizard of OZ phrase and its use in human computer interaction was coined by Dr. Jeff Kelley. He was a usability expert who was inspired by the scene in the movie “The Wizard of Oz”, where Toto reveals that the wizard is just a man behind the curtain flipping switches and pulling levers.

At the time I joined the lab, non-contact sensors were being explored (i.e., sensors that could, unlike the bumper skirt technology, identify obstacles without needing physical contact with them). Drs. Alex Mihailidis and Jesse Hoey had just won a competition for a project that used an infrared sensor to automatically detect obstacles and stop the wheelchair in the case of an imminent collision. While the results seemed promising, an issue with infrared sensors is that they work by transmitting and receiving infrared waves, which are also found in direct sunlight. So, these sensors can fail in direct sunlight, unless more advanced techniques are used (such as using specific patterns or pulses that could be used to identify whether the infrared wave entering the receiver is similar to ones being transmitted by the sensor).

Interestingly, Alex’s research group had recently struck a collaboration with the University of British Columbia. This was particularly relevant to me as I had already been offered acceptance into UBC’s Computer Science program for Graduate Studies and would be starting there in Fall 2006. Researchers there had been doing interesting work with stereo-vision sensors. The advantage of these types of sensors over those being used previously was that, much like the human eyes, not only could stereo-vision sensors figure out how far away an obstacle is (proximity), but could also be used to automatically create maps from visual landmarks. The application of computer vision to smart wheelchair research had just begun, and I was excited to be part of this pioneering work.

Over the next 6 years, I would go on to learn not only about computer vision, but also artificial intelligence, robotics, machine learning, and human computer interaction, finally bringing all these fields together in a highly trans-disciplinary PhD dissertation. Join me in my next blog posts as I share my learnings.

References:

Marcantonio ER. Dementia. In: Beers MH, Jones TV, Berkwits M, Kaplan JL, Porter R, eds. Merck Manual of Geriatrics. 3rd ed. Whitehouse Station, NJ: Merck & Co., Inc.; 2000:357-371.

Wang, R.H., Gorski, S.M., Holliday, P.J., and Fernie, G.R. (2011). Evaluation of a contact sensor skirt for an anti-collision power wheelchair for older adult nursing home residents with dementia: Safety and mobility. Assistive Technology, 23(3): 117-134.

Wang, R.H., Mihailidis, A., Dutta, T., and Fernie, G.R. (2011). Usability testing of multimodal feedback interface and simulated collision-avoidance power wheelchair for long-term-care home residents with cognitive impairments. Journal of Rehabilitation Research and Development, 48(6): 801-822.

John F. (“Jeff”) Kelley. 2018. Wizard of Oz (WoZ): a yellow brick journey. J. Usability Studies 13, 3 (May 2018), 119–124.

Viswanathan, P., Wang, R. H. and Mihailidis, A. (2013). Wizard-of-Oz and Mixed-Methods Studies to Inform Intelligent Wheelchair Design for Older Adults with Dementia. 12th European AAATE Conference, 19-22 Sept, Vilamoura, Portugal.

Mihailidis A, Elinas P, Boger J, Hoey J. An intelligent powered wheelchair to enable mobility of cognitively impaired older adults: an anticollision system. IEEE Trans Neural Syst Rehabil Eng. 2007 Mar;15(1):136-43. doi: 10.1109/TNSRE.2007.891385. PMID: 17436886.

Blog originally published on brazemobility.com