Facebook's AI research could help develop smarter AR glasses and robots, according to the study
Facebook envisions a future in which you'll learn to drums or create scrumptious meals while wearing augmented reality glasses or other artificial intelligence-enabled devices. To make that a reality, the social network needs to see through your eyes to ensure that its AI systems can see it.
"This is the world where wearable devices could assist you and me in our daily lives by providing information at the right time or helping us fetch memories," said Kristen Grauman, a Facebook lead research scientist. She stated that the technology may one day be used to analyze our actions and help us locate lost items, such as keys, that she believes may be useful in the future.
Facebook's Ray-Ban smart glasses, which debuted in September without AR effects, prove that that future is still a long way off. Part of the challenge is allowing AI systems to better understand photos and videos captured from their viewpoint so that they may assist people remember important information.
Facebook said it collaborated with 13 universities and labs to recruit 750 people to capture more than 2,200 hours of first-person video over two years. The participants, who lived in the UK, Italy, India, Japan, Saudi Arabia, Singapore, the US, Rwanda, and Colombia, shot video footage of themselves engaging in everyday activities such as playing sports, shopping, looking at their pets, or gardening. They used a variety of wearable devices, including GoPro cameras, Vuzix Blade smart glasses, and ZShades video recording sunglasses.
Beginning next month, Facebook researchers will be able to request access to this vast collection of data, which the social network claims is the world's largest collection first-person unscripted videos. The new project, Ego4D, provides a glimpse into how ioT could improve technologies like AR, virtual reality, and robotics so they can be more integrated into our daily lives.
The firm's actions come in the midst of a turbulent period for Facebook. After The Wall Street Journal published a series of stories about how the social network's internal research showed it knew about the platform'' ills even as it minimized them publicly, the business was under scrutiny from lawmakers, advocacy groups, and the public. Frances Haugen, a former Facebook product manager turned whistleblower, testified last week before Congress about the contents of thousands of pages of confidential documents she took before leaving the firm in May. She's scheduled to testify in the United Kingdom and meet with Facebook' s semi-independent oversight board in near future.
Even before Haugen's revelations, Facebook'' smart glasses sparked concerns from critics who fear the device may be used to hide people' 'doings'. During its study into first-person video, the social network said it addressed privacy concerns. Camera users could view and delete videos, and the firm blurred the faces of bystanders and license plates that were captured.
Fueling more AI research will be crucial.
Facebook stated that as part of the new initiative, it created five benchmark challenges for researchers. The benchmarks include episodic memory, so you know what happened when, forecasting, which computers know when you're going to do next, and hand and object manipulation, to see what a person is doing in filmed footage. The last two benchmarks are understanding who said what, and when, in a video, as well as who the participants are in the interaction.
Grauman said, "This sets up a bar just to get it going." "This is usually quite useful because you'll have a systematic method to evaluate data."
Helping AI understand first-person video can be challenging because computers learn from photographs taken from a spectator's third-hand perspective. When you record yourself kick a soccer ball or ride hordes of roller coasters, challenges such as motion blur and video from different angles come into play.
Facebook said it's considering expanding the project to other countries. Diversifying the video footage is important because if AR glasses are used to help a person prepare curry or do laundry, the AI assistant must be aware that those activities may be different in different areas of the world.
Facebook said the video dataset includes a variety of activities shot in 73 locations across nine countries. Participants included individuals of varying ages, genders, and professions.
The COVID-19 epidemic also created limitations in the research. More footage in the data set is of stay-at-home activities such as cooking or crafting than of public events, for example.
Some of the universities that partnered with Facebook include the University of Bristol in the UK, Georgia Tech in America, the Tokyo University in Japan, and Universidad de los Andes in Colombia.