A short review of Neural Network Algorithms(ニューラルネットワークアルゴリズムの短評)

  As COVID-19, we need to make the habit of washing our hands regularly. I bought a new apple watch 7 in spring, and my apple watch can detect when I start washing and encourage to keep going for 20 seconds. I thought it sensed that I was twirling my wrist, so it knew I was washing my hands. I mimic the hand washing action but it doesn’t show the Handwashing Timer. I was curious, and I did a lot of research. I learned that it was an application of machine learning, so I started reading some books on deep learning.

In the current deep learning boom, neural network algorithm is the most well-known and widely used machine learning algorithm. It's no exaggeration to say that the vast majority of artificial intelligence products that you can come across are using neural network algorithms, such as face unlocking, face retouching, and character recognition in photos, which are often used in cell phones, are all based on neural network classification algorithms.

1.1 Neural network algorithm features

As we know, the essence of deep learning is neural network algorithm. Theoretically, with enough data and hidden layers, neural network algorithms can fit any function. In general, neural network algorithms have the following characteristics:

1) A kind of black box algorithm.

Neural network algorithm, are also known as "black box algorithms", because there is no way to know from the outside how a neural network model is trained. For example, using a cat face recognition model with a 97% prediction accuracy, sometimes a picture of a puppy's face will be grouped into a kitten, which is unexplainable, hence the image of a neural network algorithm as a "black box algorithm".

This can also explain why banks do not use neural network algorithms to judge the creditworthiness of users because if there is a prediction error, the bank cannot trace the cause of the error and cannot give a reasonable explanation to the customer.

2) High data volume requirements.

In the 1970s and 1980s, when the Internet was not well developed, the lack of data volume was a major factor that hindered the development of neural networks. To compared with a traditional machine learning algorithm, more data (at least thousands or even millions of labeled samples) is often required to train a good neural network model.

To take the facial recognition as an example, it requires faces in various posture styles, angry, joyful, sad, wearing glasses, blurry, etc., in short, the more the better. The larger the data set, the better the performance of the neural network.

3) Demand for computing power and high development cost.

In terms of computation, neural network algorithms consume more computer resources than traditional algorithms, it would take even weeks to train a good model. However, with the level of computer hardware in the 1970s and 1980s, it was almost impossible to achieve such a large scale of computation. Therefore, the hardware performance of computers is also one of the factors that affect the development of neural networks.

After entering the 21st century, the hardware performance of computers has developed rapidly, which has created a favorable external environment for the development of neural networks.

1.2 The implementation of the Convolutional neural network Algorithms in the IOT industry

With the development of CNN, it has spawned an explosive growth of IoT devices. The IoT market continues to grow despite chip shortages and the expanding impact of COVID-19 on the supply chain. By 2021, IoT Analytics expects the number of connected IoT devices to grow by 9% to 12.3 billion active endpoints worldwide, with cellular IoT now exceeding 2 billion. By 2025, the number of IoT connections could exceed 27 billion. (※2)

Convolutional neural networks often have high data volume requirements and take up more storage space and computational resources. The embedded devices in practical applications often have limited memory space and computational power due to the consideration of power consumption, and deep learning applications cannot be well deployed in such scenarios. Based on the above analysis, I think one of the next hot types of research on deep learning will be the light-weighting for the convolutional neural networks.







 コロナの関係で、定期的に手洗いをする習慣をつける必要があります。春に新しいapple watch 7を購入しましたが、私のapple watchは手洗いを始めると、それを検知して20秒間続けるように促してくれます。手首をくるくる回しているので、手を洗っていることを察知しているのだと思いました。手洗いの動作を真似しても、手洗いタイマーが表示されません。気になって、いろいろと調べてみました。機械学習の応用だとことを知り、ディープラーニングの本を何冊か読み始めました。


1.1 ニューラルネットワークアルゴリズムの特徴











1-2  IOT産業における畳み込みニューラルネットワークアルゴリズムの実装

畳み込みニューラルネットワークアルゴリズムの発展とともに、IoTデバイスの爆発的な成長を産み出しました。チップ不足やCOVID-19のサプライチェーンへの影響拡大にもかかわらず、IoT市場は成長を続けています。IoT Analyticsでは、2021年までに接続されるIoTデバイスの数は、世界で9%増の123億アクティブエンドポイントとなり、セルラーIoTは現在20億個を超えると予想しています。2025年には、IoTの接続数は270億を超える可能性があります。