¡An AI capable of visualizing us and knowing our behavior at all times!

Source: Giphy

Greetings, here again with another publication, which I find something interesting and that's why I decided to share it with you. Without further ado, let's go to the interesting.

Well, if, as you read in the title, there is an IA capable of all that, and those responsible are in an MIT laboratory, specifically the CSAIL (Computer Science and Artificial Intelligence Laboratory), which they have called this as "RF-Action".

Perhaps at this moment, you are asking yourself the questions of How does this work, What does it use to do this follow-up, and why not others ..., well we have to start by saying that this is a deep neural network model from end to end, which (as this in the title), can detect our actions even if we are behind a wall, and even in conditions of little or no lighting present.

Source: Medium

Technical explanation: RF-Action architecture. RF-Action detects human actions from wireless signals. It first extracts 3D skeletons for each person from raw wirelesssignal inputs (yellow box). It then performs action detection and recognition on the extracted skeleton sequences (green box). The Action DetectionFramework can also take 3D skeletons generated from visual data as inputs (blue box), which enables training with both RF-generated skeletons andexisting skeleton-based action recognition datasets. Source:  The paper published in arxiv  

To achieve all this, we make use mainly of the radiation found in the frequencies of the WiFi signal (since these can cross the walls), and in this way we can generate the "3D skeletons", which serve the system to learn about the behavior and of course the system can also make use of cameras that allow you to use data based on existing visions (people in action) to complement their application with the data obtained.

Source: Medium

In the previous set of images, we can see how "RF-Action" can detect people in various environments and what their actions are at the moment. Something impressive without any doubt and even more taking into account the accuracy of them.

It is worth mentioning that this model is not the first one used for this purpose, in fact there are other recognition models such as SOTA-HCN (used for the recognition of actions based on skeletons), and SOTA Aryokee (used for the recognition of actions using radio frequencies in scenarios visible through walls). But the "RF-Action" demonstrated during the experiments to have a great accuracy, which manages to surpass the two mentioned above, so there is no doubt that it can be used in commercial applications (as in intelligent home systems, etc.).

At the moment, it is not completely finished and some improvements to the system still need to be made so that it remains at 100%, since for example the data on action detection by means of radio frequencies is still low, so it is necessary to continue feeding the system with more data, and that all these experiments have been tested in a detection range of 1 to 11 meters and with only one wall in between. Bearing in mind that (as they are sure to know from common experience) the WiFi signal loses intensity as objects (walls, furniture, etc.) are found, this is another factor that affects such a project since it uses a certain signal/noise ratio to make the "skeletons".

From an extract of the commentary made by Tianhong Li, one of the researchers of the project reveals why this technology could be used:

"For example, monitoring abnormal behaviors of the elderly, such as falling at home, monitoring whether patients take their medications properly, or remote control of intelligent home devices through actions." Source: extract from the commentary by Tianhong Li.

I personally see this type of technology as a double-edged sword, but while there are good and bad things like everything in life, the application that Li relates about the use for the elderly seems to me a genius, and this would be part of insurance in conjunction with some other technology, in potential help for them, and I'm sure it would be very beneficial to them.

Another use that occurs to me for this, is the application in security, either in conjunction with some closed circuit camera system, which would make it especially useful in places where security is a priority, or because it does not go a little further in places where it is necessary to have clear environments of certain types of objects, such as weapons, which would help prevent or alert immediately in case of shootings, or any irregular situation.

Now that they've come this far, what do you think the technology could be used for, would you be willing to use it in the future, I read them in the comments.

As always, thank you for reading me, I don't say goodbye without wishing you a happy day, evening or night.

Interesting links:

Paper “Making the Invisible Visible: Action Recognition Through Walls and Occlusions”  

Comparison of the above mentioned models  

HCN Model Paper  

Aryokee Model Paper

Fuente: Mi galería de Giphy