Peloton's use of computer vision to increase performance

Peloton's use of computer vision to increase performance ...

A television monitor monitors you as you do push-ups, squats, or ab exercises, or pull dumbbells, jump, or stretch.

You are tracked on your form and your performance in an exercise (or lack thereof); you receive suggestions on what cardio, bodyweight, strength training, or yoga workout to do next; and you earn achievement badges.

Peloton Guide, a camera-based, TV-mounted training device and system powered by computer vision, advanced algorithms, and synthetic data, provides the next-level home fitness experience.

Sanjay Nichani, the CEO of the Pelotons computer vision division, discussed the technology's development and ongoing enhancement in a livestream this week at Transform 2022.

Motivation driven by AI

Peloton's computer vision capability monitors participants' movements and provides real-time feedback. A self mode feature allows users to pan and zoom their devices to monitor themselves on-screen and ensure they are in good form.

Nichani stressed the importance of metric-driven accountability when it comes to fitness, claiming that insight and progress are both tremendous motivators.

According to him, obtaining the final Peloton Guide commercial product was an iterative process. The initial objective of AI is to bootstrap quickly by gathering small amounts of custom data and combining this with open-source data.

Nichani believes that once a model is created and deployed, thorough investigation, evaluation, and telemetry are utilized to continuously improve the system and make focused improvements.

According to Peloton researchers, the machine learning (ML) flywheel starts with data. Real data is complemented by a substantial quantity of synthetic data, generating datasets using nomenclature specific to exercises and poses together with appropriate reference materials.

Nichani described a traditional computer vision technique as pose estimation and matching, accuracy recognition models, and optical flow.

Computer vision is affected by many different variables.

Nichani said that one of the challenges of computer vision is the complexity of the applications that must be taken into account.

This includes the following:

  • Member attributes: gender, skin tone, body type, fitness level and clothing.
  • Geometric attributes: Camera-user placement; camera mounting height and tilt; member orientation and distance from the camera.

Peloton's developers conducted extensive field testing to enable edge cases and included a feature that alerts users if the camera can't distinguish them due to any number of reasons, according to Nichani.

The issue of bias

According to Nichani, fairness and inclusivity are both critical to the development of artificial intelligence models.

According to him, the first step to minimizing bias in models is ensuring that data is varied and has enough value across many attributes for training and testing.

Despite the fact that the data is broad, a diverse dataset does not guarantee impartial systems, according to the author. Bias tends to creep in, even in deep learning models, even when the data is objective.

All sourced data is tagged with attributes in the Pelotons process, according to Nichani. This allows models to evaluate performance over different slices of attributes, ensuring that no bias is detected before they are released into production.

If bias is discovered, it is addressed and ideally corrected through the flywheel process and deep dive analysis. According to Nichani, Peloton developers observe an equality of odds fairness ratio.

A classifier predicts that any label or attribute will be equally suitable for all values of that attribute.

Models were designed to account for aspects of body type (underweight, average, overweight) and skin tone in predicting whether a participant would do a crossbody curl, a squat, or a dumbbell swing, based on the Fitzpatrick classification, which, although widely accepted, has nonetheless limitations.

Nevertheless, any difficulties are far outweighed by substantial opportunities, according to Nichani. AI has many applications in the home fitness industry, from personalization to accountability, to convenience (voice-enabled commands, for example), to overall engagement.

Nichani believes that providing insights and statistics helps improve a user's performance and drives them to strive to achieve more, especially when it comes to training. Peloton tries to provide personalized gaming experiences so that youre not looking at the clock while you're exercising.

Watch the whole conversation from Transform 2022.

You may also like: