Data Science, IoT and Reinforcement Learning in High Tech Manufacturing

(By Ramakrishnan Ramanathaiah)

Modern day enterprises need to be very agile and efficient in order to succeed in today’s hyper-competitive business markets. Manufacturing is no different. Technology has revolutionised how manufacturing is done and new advances in technology are continuing to make their way in high-tech manufacturing industries in 2020 and beyond.

Particular sets of technologies which are fairly recent entrants in the high-tech manufacturing field are cloud based analytic, IoT and machine learning based techniques. These technologies can be developed, tested and deployed in a highly scalable manner using cloud computing based platforms. The internet of things (IoT) based components in manufacturing collect data and forward it to the cloud where it is consumed, processed and analysed before being used to inform business decisions. Machine learning in particular is used for classes of industrial problems where there is a lot of data and thus iterating through each and every data point and analysing every data point separately is not easily achievable.

There are three different classes of machine learning algorithms and these are i) supervised learning, ii) unsupervised learning and iii) reinforcement learning based machine learning algorithms (sometimes called semi-supervised learning). A supervised machine learning algorithm is one that takes as input a dataset that provides both the valid input and the corresponding correct valid output. The algorithm then tries and ascertain the link between the input and output to develop a mathematical model. The model can then be used for it’s intended purpose such as for prediction.

A unsupervised algorithm is a machine learning algorithm that takes a dataset with valid inputs but with no corresponding valid outputs to go with it. The algorithm therefore does not have a sample set of correct outputs which it can use to train the model but it usually has a set of rules it uses to identify correct responses. Usually unsupervised learning requires a lot more data than supervised learning if the machine learning algorithm is to be used for predictive purposes.

Reinforcement learning is a different machine learning paradigm that is mostly used for sequential decision making. It is used when the current output depends not only on the current input but also on all past outputs. It is thus of high interest in the robotics field as a robotic arm usually needs to make several sequential movements in order to perform it’s specific task. Reinforcement learning is therefore being looked at as a key technology that could revolutionise high tech manufacturing in the near future.

Reinforcement learning, for instance, can be used to predict and/or identify the sub-optimal operation of high tech equipment in a high tech manufacturing scenario. This is done by developing a model that looks at the current performance statistics of a particular machines over time and tries to predict and/or identify those which are operating sub-optimally or identify those that are likely to need repair fairly soon.

The goal of reinforcement learning is to train a machine learning algorithm to achieve a goal by outputting a particular sequence of outputs for a given sequence of output. The rule that the machine learning algorithm uses to map inputs to outputs is called the policy. It is the goal of the machine learning algorithm to randomly explore the solution space until it finds a policy that allows it to achieve it’s intended goal. This can require the algorithm to run for much longer than it would do if the algorithm in use was supervised.

Reinforcement learning is also being explored in the industrial robotics sector to try to assist industrial machines to handle industrial goods. Handling and moving an industrial goods usually involves a large number of individual movements from an industrial robotic arm. The movements are very difficult to pre-program using convectional programming techniques because of the large number of individual sequential movements required. Research on robotics powered by reinforcement learning is now being seriously explored.

Other emerging cutting edge applications of reinforcement learning includes in the allocation or subdivision of computing resources between many different industrial machines. It is difficult to do this manually because of the sheer number of different ways in which computing resource can be divided. This is especially true if the subdivision needs to be continual (i.e. the resources allocated to each machine keeps changing in time). Reinforcement based algorithms have been shown to be effectively and determining optical allocations of resources by inferring an optimal policy from past allocations.

In conclusion, reinforcement learning can be used with the help of the cloud platforms to improve efficiency in modern enterprises. In particular, it can be used in cases where sequential input needs to be used to provide insights into the future performance of enterprise systems in high tech enterprises.

(The author is the Director- SAP Practices, Miraclesoft)