The Accuracy of Wearable Technology

The rise of wearable devices in the consumer wellness market can feel overwhelming at times. So too can the number of digital biomarkers that companies claim to accurtely measure from these devices – such as sleep cycles, heart rate variability, recovery, temperature, respiratory rate, emotions, body fat, and many more. If we forget the marketing for a moment, it is important to ask two questions when you invest in a wearable or digital health:

1) Has the technology been validated in evidence-based medicine? In other words, is the device accurate according to impartial scientific studies, not just company-sponsored studies?

2) Does the wearable serve your goals? As more digital biometrics hit the market it is important to stand back from the noise and make sure it works for you. Measuring more physiological variables is not always better. Sometimes, less is more.

In this article, we’ll help you answer both these questions. Every technology has pros and cons (including 360), and we hope to empower you with objective information about emerging health technologies, and the rationale behind ours.

Heart Rate Variability (HRV) 

Various devices claim to able to measure heart rate variability (HRV) from the wrist. HRV is a marker of parasympathetic tone. This is the part of the autonomic nervous system responsible for regeneration, repair, and rebalancing of systems.

HRV has traditionally been measured by electrocardiography (ECG), which measures the electrical activity of the heart. The gold standard is to use a device like a Holter device that uses electrodes and electrical leads to measure moment-to-moment changes in the variability in the time between each contraction of the heart.

ECG can compute multiple different metrics beyond heart rate, including SDNN, RMSDD, AVNN, PNN50, LF, HF, LF/HF Ratio, ULF, and VLF. These different heart rate variability metrics have been studied extensively in clinical research across health and disease and can provide very powerful insight into what keeps us healthy and makes us unwell.

Using a traditional ECG device is impractical for us in our day-to-day life – they are expensive, cumbersome, and only provide raw data. In recent years, new technologies have emerged claiming to be able to quantify HRV and give us insight on the back of it.

What does the science say?

Polar H10 Heart Sensor Vs ECG

Let’s begin with the device you use – the Polar sensor. The Polar H10 uses electrodes built into the strap alongside the sensor to capture the electrophysiological activity of the heart and quantify ECG status.

Independent research has tested Polar against gold standard ECG devices. You can see the results of this research in the image below:

The study, published in the European Journal of Applied Physiology, found a near-perfect correlation between the Polar H10 and Holter device (r = 0.997, p > 0.001).

In this study, the Polar H10 was validated against Holter ECG not just at rest, but also during exercise. Comparing accuracy during different types of exercise (household, walking, jogging, strength training) is useful because with more vigorous activity there is a greater chance of signal errors, noise, and interference called movement artefacts.

The study concluded that the Polar H10 was equal to Holter, and should be regarded as an equivalent gold standard in the measurement of ECG status and data for heart rate variability.

This research should provide you with genuine peace of mind. With independent validation against gold-standard ECG, you can be confident that when you check-in on 360, the objective data is clinical-grade quality.

Wrist wearables Vs ECG

When it comes to HRV and wrist wearables, the first and most important thing to say is that these devices cannot directly capture electrophysiological ECG data from the heart because they do not use electrodes close to the chest, like Holter and the Polar. Thus, when any company claims they measure HRV from the wrist, strictly speaking, they are not.

Because wrist wearables cannot directly capture HRV they use a surrogate called Pulse Rate Variability (PRV), but then refer to it as HRV. Pulse Rate Variability is captured using photoplethysmography (PPG), which involves using sensors to shine a light onto an area where capillaries are easy to access. The light is then reflected back to the sensor to depict blood volume in the vessel and thus forms the grounds of a heartbeat.

It’s worth emphasising again: pulse wave variability is not the same as heart rate variability. They are different physiological events. New research this year suggests that PWV and HRV are a distinctly different phenomenon in the body and should be treated as different biomarkers. Nevertheless, consumer wearables claim to measure HRV accurately off the wrist.

What does the research show when comparing wrist wearables to gold-standard ECG?

The scientific evidence is still very mixed, and for this reason, 360 does not use wrist wearables for the measurement of biomarkers including HRV nor recommends them.

Part of the problem here is that much of the research coming out by various wrist wearables is sponsored by the manufacturers themselves. This is normal in the sense that emerging technologies want to validate themselves, and in essence, have to sponsor early studies using their technology. But funding a study naturally makes it open to bias, so cannot be included in any final analysis at this point.

Independent research suggests wrist wearables can measure pulse wave variability accurately generally in just young and healthy populations at rest. For example, a recent study measured PRV versus clinical-grade HRV across different population types (from young and healthy to older and unhealthy).

The researchers only found strong agreement between PRV and clinical-grade HRV numbers in a small group of the population (those in their early twenties who were lean, fit, and healthy). For everyone else, it was not accurate. As you can imagine, this is a problem for most of us.

A recent study stands out for measuring pulse wave variability in wrist wearables in emergency physicians for a year. The goal of the study was to see if PPG data could help understand and prevent burnout in these workers by understanding changes in physiology over time.

A lot of studies measure PPG or HRV for a short-time: one night, one day, or one week. Therefore, as a longitudinal study, the design of this research study was important.

All-in-all more than 400 hours of PPG data was collected over the year in an attempt to understand changes in physiological well-being captured by pulse rate variability. Upon analyzing the data, researchers found that only 8.54% of data was interpretable. Put another way, 91.46% of data was unusable, such were the level of inaccuracies in the data.

The authors of this study concluded:

“Although the use of PPG biosensors to record real-time physiological data from emergency physicians while providing clinical care seems operationally feasible, this study fails to support the notion that such an approach can efficiently provide reliable estimates of metrics of interest.”

In other words, using PPG from wrist wearables to measure HRV is prone to errors. If we are to make daily decisions in our health based on data, we want the margin of error from that data to be as small as possible. This is why we do not use wrist wearables.

Rather than just cite the mixed evidence for wrist wearables, it is important for you to know why they can present challenges in terms of getting accurate data to drive daily behaviours.

Challenge 1: Poor Data In/Poor Data Out

Technology is only as good as the raw data coming in. The old saying of “junk in, junk out” is as true for digital health as any other form of technology.

There are a few reasons why wearables are at a disadvantage compared to heart rate sensors. The first reason is the errors caused by movement, change in pressure, change in light, change in temperature, and position of the wearable on the wrist. These issues can cause a sudden change in raw data. Just a single error can cause a major change in your HRV score. This is likely to be one of the reasons why in the physician study above, over 90% of data had to be excluded.

Second, because wrist wearables use light to measure changes in the waveform of pulse rate, human factors such as tattoos, skin colour, and hair on the wrist can be potential factors interfering with the ability of the sensor to collect high-quality raw PPG data.

Research suggests emotional stress can also cause errors in PPG pulse wave variability data. If we are to use metrics like HRV to measure how we are adapting to physical and emotional stress, we need to make sure it is capable of doing so successfully.

Challenge 2. Filtering Data Inappropriately

The second main issue with wrist wearables is understanding how manufacturers deal with data errors. What happens when you have several errors in raw data in a reading?

There are key recommendations from clinical research to guide digital health companies in identifying and rectifying ectopic heartbeats, movement artefacts and errors in HRV readings, and 360 follows these established recommendations.

Unfortunately, because of movement artefacts and noise unique to each wrist wearable, companies often have to add multiple in-house filters to deal with the problem, which aren’t based on established recommendations. As a result, they move away from evidence-based clinical guidelines. A recent study in the prestigious journal Nature found that it was not possible to determine the accuracy of data filtering in any wrist wearable, apart from one.

Challenge 3. Interpretation of the Data

If a company measuring HRV overcomes the first hurdle of accurate data collection and the second hurdle of accurate data filtering, there is a third and final challenge: understanding accurately what that final data point/score means. This is where so many digital developers and wearables companies go wrong.

For example, you will see companies tell users that low HRV is bad and high HRV is good. This is grossly simplistic. For a start, HRV tends to declines with age, so your score is relative. A low score for someone might be a great score for you, depending on your age.

Because age is one factor that impacts an understanding of your score, we give you the choice of putting your year of birth into 360 in the app. This enables you to see in the objective trends graph how your HRV compares to others in your decade of life (20’s, 30’s, 40’s, etc). We do not ask for your full date of birth because 360 is built on trust and privacy, and we want to ensure you remain anonymous at all times.

Second, because physical and mental health is a factor that impacts scores, we give you bands within those age ranges so that you can begin to see how your trendline compares to “average”, “good” and “excellent” scores of those in your age range. Once again, the goal here is to help you understand population norms. It is all too common to see companies tell users that their scores are so personalised they cannot be compared to anyone else. This is untrue. Like our scoring between 0-10, those bands we put in the objective trends graph are based on high-quality ECG-derived HRV data from tens of thousands of people from evidence-based medicine. It’s meaningful data.

Third, sometimes a low score can be a good sign, and sometimes and a high score can be a bad sign. For example, on days of competition, or in the run up to a competition, it is normal to see an elite athlete’s HRV drop. This is normal, and often very healthy. Peak performance requires sufficient activation of our sympathetic nervous system. If our HRV scores were really high on the day of competition, it may indicate that the sympathetic is not sufficiently active enough to mobilise resources in the body and brain to make the best decisions possible, such as adrenaline and noradrenaline that contributes to focus and attention. Stress can be a good thing for high performance.

Equally, high scores can be a bad sign. On occasions, people notice their HRV spikes up even though their lifestyle has not been that healthy and they feel tired, inflamed and/or under the weather. This spike can happen when the parasympathetic pushes recovery hard in an attempt to dampen inflammatory activity and rebalance systems. In this instance, a high score is not a green light to go train hard. It is a time to focus on regenerative lifestyle factors to bring the body back into autonomic balance.

Because low scores are not always bad and high scores are not always good, we built the 0-10 objective scoring range to be in line with population norms from the research. We also built a whole system behind the scenes powered by AI and machine learning that uses all the inputs you provide at check-in to understand how you are doing, the trajectory of your wellbeing, and what 360 can do to guide you personally based in your data.

We do not believe in technology for the sake of technology. We built 360 to make it easier for you to protect your health, energy, and performance.

At every step, we have done this with a view to making your digital health experience as simple, non-invasive, and accurate as possible.

This is why we ask you to use a sensor for 90 seconds each morning, and not ask you to wear a 360 wrist wearable all day.

This is why we focus on core metrics that do matter, rather than dozens of others that don’t.

This is why we anonymise your data completely, unlike technology companies that capture several layers of identifiable information.

No technology can be 100% accurate, all of the time. That includes 360. But we do our best to ensure that anything you read, touch, or experience with 360 is based on brilliant science from the incredible work of independent researchers all over the world, not just our own.

Over 2800 research papers go into what you experience with 360 – all with one aim – to help you make small improvements in your daily life that have a positive compound effect down the line.

In the excellent book Atomic Habits, author James Clear highlights the importance of small changes and the value of getting 1% better every day. You can see this in the image below:

In the book, James Clear puts it very eloquently when he says:

“It is so easy to overestimate the importance of one defining moment and underestimate the value of making small improvements on a daily basis. Too often, we convince ourselves that massive success requires massive action. Meanwhile, improving by 1 percent isn’t particularly notable—sometimes it isn’t even noticeable—but it can be far more meaningful, especially in the long run. The difference a tiny improvement can make over time is astounding.”

Use 360 to provide you with the proactive and personalised insight you need to make those tiny improvements each day. We look forward to hearing about your success.