Machine Learning has given us the opportunity in recent years to automate things that were not possible before. Computer Vision uses Machine Learning to recognize objects in images. We no longer need to write an algorithm for a task specifically. Instead, we can train a Machine Learning model to perform the task by providing examples.
Within road construction, we see that companies also want to innovate and, as a result, get started quickly with Machine Learning. A recent development is the use of Machine Learning to detect defects in road surfaces , such as potholes and cracks. To realize this, it is necessary to train a model before deployment, by feeding it images together with image annotations (human interpretations). THe model then learns to recognize where defects are by comparing its own output to the provided annotations, allowing it to improve itself. This is repeated until the model reaches a sufficient level of performance and it no longer needs the annotations. The model can then be used in the real world to inspect roads.
The learning process behind the Machine Learning model is based on several assumptions about the data, the most important of which is the assumption of independent and identically distributed data. (i.i.d.) The i.i.d. assumption has made many developments possible in Artificial Intelligence. However, it doesn't always hold in the real world. If the probability distribution changes, then the models based on the i.i.d. may not be reliable.
For a model that depends on the i.i.d. assumption, we don't always know what will happen in practice. It can continue to work, but it can also fail in unexpected ways. An i.i.d. assumption consists of two parts, the independent part, and the identically distributed part. The independent part assumes that observations do not influence each other. This eliminates the need for the model to remember a prediction based on an image to make a prediction for another photo. On the other hand, things often go wrong with the identically distributed variable. This part implies that real data has the same distribution as the training data. As a result, models often make false predictions because real-world data constantly changes due to various factors. Machine Learning models, therefore, need to constantly observe, learn from the environment, and adapt their behavior to changes.
For example, an image of a road in Italy comes from a different distribution than a Dutch road. The differences can vary from the type of asphalt used to the average ambient temperature. A distribution can be affected by all of these factors and more. Moreover, it is impossible to measure whether two pictures come from the same distribution. This makes it impossible to check the assumption. Imagine building a forklift without being able to control how much weight it can carry. Although we cannot measure the actual distribution, it is often possible to say something about the difference in distribution between the two images. The same applies to roads. Images of roads in Italy will come from a different distribution than images of roads in the Netherlands. What happens if we train a model in the Netherlands and start using it in Italy? It probably won't fail completely because cracks from other countries are the same as those in the Netherlands. However, the model will undoubtedly contain more errors due to external factors.
To solve this problem, new training data will have to come from Italy. Even if you have already made thousands of annotations in the Netherlands, thousands of photos with defects from Italy will have to be annotated. A more efficient approach is to replace the existing i.i.d. assumption with a new one. After all, some characteristics will remain invariant regardless of the change in distribution. To achieve this, multiple domains will have to be used to retrain the model. In this case, we tell the model which domains there are, and apply a method such as Invariant Risk Minimization (IRM) or Causal Matching (CM).
These methods work as follows. The model collects a number of characteristics that it uses to identify certain defects, such as cracks. It keeps track of which attributes it uses in each domain. In South Holland, for example, the inside of a crack in the road is black, while in Groningen, it is light gray. However, the shapes of the cracks are similar. By comparing the two, the model sees that it should not pay attention to the color as this is a changeable characteristic. Instead, it should only use the form. When we deploy this model in Italy or Norway, it has learned to use only the crack shape and work just as well there.
In short, Machine Learning is playing an increasingly important role in performing human inspection tasks. This development will also have a major impact on road construction and provide many new opportunities to make roads better and safer. However, Machine Learning is not just about collecting data. Although more data generally gives better results, knowledge about the data and choosing the right domains is just as important. Then by including information about the distribution, as we do when using IRM or CM, a model can generalize to never-before-seen situations with fewer data. In this way, companies can handle data more efficiently and get more out of what Machine Learning has to offer.
Read the full dutch article here
Authors: Daniël Nobbe, Research Intern at BrainCreators and Maarten Stol, Principal Scientific Advisor at BrainCreators
Are you interested in more in-depth information about this subject and our solution? Download our free ebook: