Topic / News

Microsoft and Novartis collaborate on MS assessment tool

When Microsoft launched the Kinect for X-Box six years ago, it set a Guinness World Record for the fastest-selling consumer device ever. But while gaming households the world over were using it to serve up virtual aces as the sensor tracked their movements in the comfort of their lounge-come-tennis-court, the device had caught the attention of the multinational, healthcare giant Novartis

When Microsoft launched the Kinect for Xbox six years ago, it set a Guinness World Record for the fastest-selling consumer device ever.

But while gaming households the world over were using it to serve up virtual aces as the sensor tracked their movements in the comfort of their lounge-come-tennis-court, the device had caught the attention of the multinational, healthcare giant Novartis.

Application to MS

As is the nature of MS, the symptoms that one patient reports may follow a very different path from those of another, and there was no particularly reliable method for quantifying whether treatments were proving effective.

One of the most used methods is the Kurtze Expanded Disability Status Scale (EDSS) which entails a series of tests (touching your nose, testing your balance etc), on the completion of which the neurologist is able to assign a score, and then refer to a scale which gives a rating for the symptoms and how they are progressing.

However, due to the very human application of this test, differing results could often be reported by different doctors. Microsoft itself noted, “The researchers found that when a groups of doctors are shown the same patient doing the same movement, some may interpret it as a 1 on the rating scale, while others will say it’s a 2.

Even when the same doctor is shown the same movement on two different days, that doctor may give that patient a different rating.”

Supporting clinical assessment

Looking for an alternative to the variability of different doctors’ assessments therefore, Novartis launched a research project called Assess MS which it says is a collaboration among Novartis Pharmaceuticals, three hospital clinics, and the Microsoft Machine Learning and Perception Research Group. 

The purpose is to design a system to support clinical assessment of MS using depth-sensing computer vision with the project’s objective to provide a more consistently quantified measure of an MS patient’s motor ability.

Abigail Sellen, principal researcher at Microsoft Research, said: “We saw a real possibility to use all of the expertise that we have in the building around machine learning and computer vision to assess and diagnosis the progression of Multiple Sclerosis. We're not out to replace a neurologist or the nurse. What we're trying to do instead is provide some data which will help the expert make a better assessment.”

It is acknowledged as a very ambitious project, entailing the development of entirely new methods of training machine learning systems, which require massive amounts of data.

Researchers would also have to enhance the ability of the depth camera (currently able to recognize big movements such as someone playing a game of tennis) to a level where it was detecting tiny changes in the subject, such as the sway of their torso, which indicate even the small progression of MS symptoms.

“We had to think hard about better, more accurate algorithms that would really pick up on the subtle signals that differentiate between an MS patient and a healthy individual,” added Antonio Criminisi, a principal researcher in machine intelligence and perception at Microsoft’s UK laboratory.

Further challenges

Further challenges have been faced, such as the practical application of this test, which requires installing a Kinect-based system into what are typically small clinical examination rooms quite often full of furniture.

Consequently, alerts were installed to inform the clinician if anything was blocking the camera; and simple additions such as wheels mean that the device can be easily moved to a less crowded position.

Microsoft now reports that following years of development work in design and algorithm, a proof-of-concept is finally ready to be tested on a small number of patients, to confirm its efficacy in principle.

Assuming positive results, Assess MS would then be tested in practice across a broader patient trial. And if successful, this could enable Novartis and other healthcare companies to refine clinical trials for MS treatments, develop better therapies and patient care, and possibly extend this to other neuromuscular diseases too.