Unsupervised

unsupervised

In this article, I'm going to discuss the topic of AI adoption. I'm going to talk about a couple of ways that organizations and companies can adopt AI more easily. More specifically, I'm going to talk about two areas of machine learning which I believe can be very easily integrated into existing systems and infrastructures, independent of the industry or the company, and can very quickly add value. Risks Behind AI Adoption One of the challenges behind AI adoption is that there are significant risks and this makes many companies hesitant to adopt AI. One of the major risks is obviously the risk of hiring. If you hire the wrong person or people, then you will have incurred a huge waste of time and money. Another issue is that a company might not be ready for the adoption of AI in terms of the culture. Unless the senior executives buy into AI, but also the middle layers, then it's very unlikely that AI can work well. And this is one of the topics that I've discussed and talked about a lot in my work, also in my book . And unless a company has the right culture for AI, it's very likely going to fail AI adoption. So that being said, companies might be hesitant to adopt AI, and then this creates a culture of fear and the problem perpetuates itself. The Decision Makers Handbook to Data Science Some other risks are obviously the risks of breaking things, right? So, if you're a company that has been going on for 20 or 30 years, maybe you're running some legacy systems and rebuilding these systems in a way that they can integrate with AI. Whether we're talking about a product or developing another platform internally in one way or another, there might not be a huge incentive to make these changes because you're afraid you must break things. And this is why, if an organization is very defensive around the adoption of AI, it's very important to start with some which are easy to understand, they're easy to implement, and the management, upper management, and mid-level management can easily understand and they support. use cases That's why, unfortunately, many AI initiatives fail; whether we're talking about products or services because, sometimes, service providers or product developers pitch ideas. Companies that might seem a little bit too far out there. Maybe the valuation is not clear, or maybe adopting these technologies would require a significant change in the culture or the infrastructure layer of a company. Two Types of Technologies There are two types of technologies that I believe can very easily add value without much effort. And these two technologies are: Anomaly Detection The term anomaly detection refers to a class of methods and algorithms in machine learning which are used to identify points that are, in a sense, unusual. These are points that do not seem to follow the regular patterns of behavior. And the thing with anomaly detection is that it can be very useful in contexts like fraud but can also be used in other domains like manufacturing. You can use anomaly detection to detect whether some machine is not behaving as it should and then act against this. Maybe this machine is about to break down, or similarly in retail, maybe you want to use anomaly detection to detect spikes in demand or to detect issues with the supply chain. The benefits of anomaly detection are that it doesn't really require that much to integrate with existing systems. In many cases, you can just take data as they are and it's very easy to just feed them into algorithms. And the thing is those anomaly detection algorithms, require fine-tuning, but they do not require training. So, one of the main roadblocks to using AI, in many cases, is the absence of what we call labeled data. So, these are data points that have a target variable. So, if we're talking, for example, about images, then labeled data is data that has been labeled as “this is an image of a human”, “this is an image of an animal”, et cetera. So, anomaly detection doesn't require labeled data, which means that it is much easier and faster to roll out than . Obviously, supervised learning is still king in terms of the applications of machine learning, but even if it's king, it's not always feasible to use it. supervised learning That's why anomaly detection can be so powerful if we're talking about early AI adoption Unsupervised Learning On a similar note, the second family of techniques and methods that I’d like to discuss is . Anomaly detection can also be seen as a subset of unsupervised learning. unsupervised learning But unsupervised learning also includes the topics and methods like . And clustering is one of the most famous cases and applications of data science. clustering So, for example, user segmentation is a very famous application, or pretty much any kind of segmentation. And I believe that the combination of dashboards, which can help an organization extract useful insight, and the addition of some basic unsupervised learning and clustering on this dashboard can be a very good way to educate the management of the uses of AI and help them better understand how AI can add value in an organization. Again, the benefit of using unsupervised learning is that you don't really require labeled data, you only need data that is structured in one way or another. But obviously, in some cases, you can also deal with structured data like text, as long as it's digitized. But the important thing is, much like anomaly detection, you don't really need to spend time leveling cases. So that's very powerful because you can get a useful system and, at the same, time you can get decision-makers within a company. They can get used to AI. And this is going to obviously help the gradual adoption and improve the understanding of what AI can do and how it can be used. Therefore, if a company is quite risk averse in that sense, I think that using these techniques, these methods can be a very good way to start with AI adoption. Conclusion - The Best Option for AI Adoption There are many other ways to do AI adoption. And in this article, I'm just referring to two methods that can be used by risk-averse organizations. Obviously, if an organization wants to move faster, then the best option by far is to go for supervised learning.

Ai Adoption: What You Need to Know About Anomaly Detection and Unsupervised Learning

As the American Society of Quality , many organizations have quality-related costs of up to 40% of their total production revenue. A large part of this cost comes from the inefficiency of manual inspection, which is the most common way to provide quality control in manufacturing. reports The application of artificial intelligence for quality control automation presents a more productive and accurate way of doing a visual inspection of production lines. However, traditional machine learning methods present several limitations to how we can train and utilize models for defect detection. So in this article, we’ll discuss the advantages of unsupervised learning for defect detection, and elaborate on the approaches MobiDev uses in our practical experience. What is AI defect detection and where it is used? is based on computer vision that provides capabilities for automating the whole process using machine learning algorithms. Defect detection models are trained to visually examine items that pass the production line and recognize anomalies on their surface, and spot inconsistencies in dimensions, shape, or color. AI defect detection AI quality inspection The output depends on what the model is trained to do, but in the case of defect detection, the flow typically looks like this: https://www.youtube.com/watch?v=UY6xbrcViVw Applied to quality control processes, defect detection AI is efficient at inspecting large production lines and spotting faults even on the smallest parts of a final product. This relates to a large spectrum of manufactured products that may contain surface defects of different natures. Intel describes a case of implementing computer vision to automate tire quality inspection. As is stated in the , the accuracy of quality control rose up from 90% to 99% percent, while approximately $49,000 in labor costs were cut by the production line. report But such systems are not bound to stationary hardware in the factory. For instance, drones with cameras can be used for inspecting or other outdoor surfaces, which significantly decreases the time required for covering large city areas. pavement defects The pharmacy industry also benefits from inspecting manufacturing lines for different products. For instance, Orobix applies defect detection to with a specific type of camera that can be used by an untrained human operator. The same principle is applied to inspecting defects, like cracks and air blobs caught in the glass. drug manufacturing pharmaceutical glass Such examples can be found in the food industry, textile, electronics, heavy manufacturing, and other branches. But, there are some specific problems in how we can approach defect detection algorithms with traditional machine learning. As manufacturers inspect thousands of products daily, it becomes difficult to collect sample data for training, and also to label it. This is where unsupervised learning comes into play. What is unsupervised learning? The majority of machine learning applications rely on methods. Supervised learning entails that we provide ground truth information to the model by manually labeling collected data. supervised machine learning In terms of a production line, collecting and labeling data can be impossible, since there is no way we can gather all the variations of cracks or dents on a product to ensure accurate detection by the model. Here we face four problems: difficulties with obtaining a large amount of anomalous data possibility of a very small difference between a normal and an anomalous sample a significant difference between two anomalous samples inability to know in advance the type and number of anomalies algorithms allow you to find patterns in a data set without pre-labeled results and discover the underlying structure of the data where it is impossible to train the algorithm the way you normally would. Unsupervised machine learning Unlike supervised learning, the flow of training becomes less labor intensive, as we expect the model to discover patterns in data with a higher threshold for variations. Anomaly detection reveals previously unseen rare objects or events without any prior knowledge about them. The only available information is that the percentage of anomalies in the dataset is small. Concerning defect detection, this helps to resolve the problem with data labeling and collecting huge amounts of samples. So let’s see how unsupervised learning methods can be used for defect detection model training. How does unsupervised learning apply to defect detection? Defect detection relates to the problem of anomaly detection in machine learning. While we don’t rely on labeling, there are other approaches in unsupervised learning that aim at grouping data and providing hints to the model. is grouping unlabeled examples according to similarity.  Clustering is widely used for recommender engines, market or customer segmentation, social network analysis, or search result clustering. Clustering aims to observe frequently occurring patterns, correlations, or associations from datasets. Association mining are designed to model the distribution probability with latent variables. It is mainly used for data preprocessing, reduction of features in a dataset, or decomposing the dataset into multiple components based on features. Latent variable models Uncovered patterns with unsupervised learning can be used for implementing traditional machine learning models. For instance, we might apply clustering to the available data, and then use those clusters as a training dataset for supervised learning models. Concrete Crack Detection with Unsupervised ML Having a vast experience in machine learning, we’ve conducted an experiment using the . The goal was to create a model capable of recognizing images with defects and normal ones using unsupervised learning. Concrete Crack dataset Additionally, the study checks how the number of defect images affects certain algorithms used in this project. In the use case we selected, we assume that image labels cannot be known in advance during training. Only a test dataset is labeled in order to verify the quality of model prediction, as training occurs by an unsupervised approach. So here we used five different approaches to get classification results from an unsupervised learning model. CLUSTERING Since we don’t have any labeled ground truth data, grouping unlabeled examples are done with clustering. In our case, there are two clusters of images we need to single out from the dataset. This was performed with a pre-trained VGG16 convolutional neural network for feature extraction and K-means for clustering. What clustering did here, is group images with and without cracks based on their visual similarities. In a nutshell, clustering looks something like this: Clustering methods are easy to implement and usually are considered as a baseline approach for further deep learning modeling. BIRCH CLUSTERING Upon this approach, images were clustered based on visual similarity with a pre-trained ResNet50 neural network for feature extraction and for clustering. This algorithm constructs a tree data structure with the cluster centroids being read off the leaf. Birch It is a memory-efficient, online-learning algorithm. The results of clustering were visualized with : Principal Component Analysis As we can see, birch clustering shows a pretty good distribution of classes, even on points where the sample is quite far from its centroid. CUSTOM CONVOLUTIONAL AUTOENCODER Custom convolutional autoencoder contains two blocks: encoder and decoder. It helps to obtain features in the encoder part and reconstruct images from them in the decoder part. As we don’t have labels for network training, we need to choose another approach to obtain classes – for example, an adaptively selectable threshold. The purpose of an adaptively selectable threshold is to divide two distributions (crack-free images and images with cracks) as precisely as possible: DCGAN DCGAN generates images from z-space with the help of adversarial loss (BCALoss). Finally, we have three losses – generator loss, discriminator loss, and MSE loss (to compare generated images and ground truth). We can build our classification on the same approach as in custom autoencoder – through comparison between losses on images with cracks and without with the help of an adaptive selectable threshold. For threshold, it will be appropriate to use discriminator loss or MSE loss depending on their distributions. GANOMALY uses the conditional GAN approach to train a Generator to produce images of the normal data. During inference, when an anomalous image is passed, it is not able to capture the data correctly. GANomaly It leads to poor reconstruction for defected images and good reconstruction for normal ones and gives the anomaly score. How to approach unsupervised anomaly detection Perhaps, the most beneficial side of unsupervised learning techniques is that we can avoid gathering tremendous amounts of sample data, and labeling it for training. Applying unsupervised learning techniques to derive data patterns, we’re not limited to which model can be used for actual classification and defect detection. However, unsupervised learning models are suited better for segmenting existing data into classes, since it’s quite difficult to check the model prediction accuracy, especially without a labeled dataset. Written by , AI/ML Team Leader at . Viktoriia Akhremenko MobiDev The full article was originally published and is based on MobiDev technology research. here

How Unsupervised Learning Can Help in Defect Detection & Quality Control in Manufacturing

There won’t be any business insights if the data quality is poor. When preparing data, I often go through many different approaches to reach a level of quality of data that can provide accurate results. In this article, I describe how unsupervised ML can help in data preparation for machine learning projects and how it helps to get more accurate business insights. What’s Wrong with Traditional Data Preparation Approaches? For accurate predictions, the data must not only be properly labeled, de-deputed, broad, consistent, etc. The point is that the machine learning model should process the “right” data. It is not entirely clear what are the criteria of the “right” data. When data scientists prepare the data, they often require a domain expert assisting in data categorization and labeling. If taking into account the majority of machine learning projects, there are often no domain experts available for such tasks. Here, the correctness of data may fail due to the wrong understanding of the final output expected from the ML model, incorrect categorization of data, and human error. In such a case, the ML model might output the wrong result at the very beginning and cause further errors in the future. One more question is whether the expected result is really something that might add value when optimizing business processes. What if the initial hypothesis is wrong? Here I mean cases when there is some underlying structure of data that neither product owner nor data scientists know about. Anyhow, data scientists spend up to 80% of their time cleaning the gathered data before training the ML model, which is not a guarantee of the entire absence of errors and bias. That is why it is often difficult to reach the ideal data quality and meet all the data standard requirements. And that’s how we came to unsupervised machine learning approaches. What is Unsupervised Machine Learning? Before reviewing unsupervised machine learning, let’s define what supervised learning is. According to , supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs. It infers a function from labeled training data consisting of a set of training examples. These labeled training data is useful for the ML model since then it differentiates data categories more accurately. The measure for the model’s performance, in this case, is defined from the beginning, and it is clear what can be considered as an “accurate” output. The possible consequences of such an approach are described in the previous paragraph. Wikipedia In unsupervised machine learning, there are no data labels outlined, and we cannot measure the performance of an algorithm. The algorithm detects a deep structure of data on its own and distributes a dataset into different categories. The algorithm is commonly chosen according to business goals, and if performed correctly, the business solution is tenfold more powerful than a supervised learning-based one. The unsupervised ML algorithm does not have predefined tasks and may detect hidden data patterns that we may not have known about. Also, it’s better to use unsupervised ML when aiming to find new patterns in future data, thus, optimizing the ML-based solution in advance. Unsupervised ML Approaches to Define the Underlying Structure of Data Unsupervised approaches are mostly focused on the finding of the underlying data structure. Here I describe the most often used approaches. DIMENSIONALITY REDUCTION ALGORITHM Dimensionality reduction is the ML method used to identify patterns in data and deal with computationally extensive problems. This method includes a set of algorithms aimed to reduce the number of input variables in a dataset by projecting the original high-dimensional input data to a low-dimensional space. Dimensionality reduction is helpful when working with visual and audio data involving speech, video, images, or text, and also when simplifying datasets in order to better fit a predictive model. There are many examples of dimensionality reduction algorithms, but I’d like to emphasize UMAP, which we found useful when working on one of the machine learning projects. algorithm allows the construction of a high dimensional graph representation of data and further optimization of a low-dimensional graph to be as structurally similar as possible. UMAP (Uniform Manifold Approximation and Projection) The main feature of this algorithm is the nonlinear representation of data. Compared to other dimensionality reduction algorithms, it is good at scaling dimensionality and size of a dataset and fast projection. For , UMAP can project the 784-dimensional, 70,000-point MNIST dataset in less than 3 minutes, compared to 45 minutes for scikit-learn’s t-SNE implementation. example UMAP projection to various toy datasets, powered by umap-js Source DATA CLUSTERING Clustering allows defining a structure in a set of unlabeled data. Simply put, it organizes data into groups (clusters), basing on its similarity and dissimilarity. This approach is applicable for market and customer segmentation, recommendation engines, document classification, fraud identification, and other cases. The goal of clustering is to detect distinct groups in an unlabeled dataset, where the users are expected to determine the criteria of what is a “correct” cluster so that clustering results meet their expectations. The example of Clustering algorithm in action There are many clustering algorithms, including KNN, K-means, Gaussian Mixture Model, and others, but here I’d like to emphasize two algorithms that have proven their high performance in our practice: DBSCAN and HDBSCAN. algorithm takes all instances that are close to each other and groups them together, based on a distance measurement and a minimum number of instances specified by a data science engineer. Since the distances within multiple clusters are specified, clusters will contain those instances which are most densely located. If some instance is located out of the specified distance, and not belongs to any cluster, it will be labeled as an outlier. DBSCAN (density-based spatial clustering of applications with noise) DBSCAN clustering visualization: Smiley Face Source This algorithm is good at finding those data structures and associations that can be useful to define patterns and predict trends. For example, when developing recommendation engines, the DBSCAN algorithm can be applied to customer databases in order to find the most demanded products of specific users. converts the DBSCAN into a hierarchical manner and extracts a flat clustering based on the stability of clusters. HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) algorithm HDBSCAN clustering algorithm Source Firstly, the HDBSCAN algorithm changes the space according to the density or sparsity, creates the minimum tree of the distance weighted graph, and constructs a cluster hierarchy of connected components. Based on the minimum cluster size, the algorithm condenses the cluster hierarchy and, finally, extracts the stable clusters from the condensed tree. ANOMALY DETECTION ALGORITHM The anomaly detection algorithm is not independent itself. It often goes along with the dimensionality reduction and clustering algorithms. By using the dimensionality reduction algorithm as a pre-stage for anomaly detection, we, firstly, transform high-dimensional space into a lower-dimensional one. Then we can figure out the density of the major data points in this lower-dimensional space, which may be identified as “normal.” Those data points located far away from the “normal” space are outliers or “anomalies.” The clustering algorithm is often used to improve the analysis of anomalies. It allows grouping of similar anomalies and further manual categorization based on their behavior types. By following such a process, we can use unsupervised ML algorithms to detect anomalies, cluster them, provide labels to each cluster manually; thus, getting business insights we couldn’t have predicted. A good example of anomaly detection is spam filtering, where the ML algorithm analyzes all incoming messages, clusters them, and detects spam messages as outliers. It is also extensively used for fraud detection in financial, insurance, IT, and other spheres. The example of Anomaly Detection algorithm Source This algorithm is also a part of ML-based predictive maintenance systems in manufacturing. By detecting anomalies in the system behavior, this algorithm allows the prediction of equipment failures before they occur. ASSOCIATION MINING ALGORITHM Association mining is the unsupervised ML algorithm used to identify hidden relationships in large datasets which frequently occur together. Compared to most ML algorithms that process numeric data, association mining can deal with non-numeric, categorical data, which means that it requires more actions than simple counting. This algorithm is commonly used to identify patterns and associations in transactional, relational, or any similar database. For example, it is possible to build the ML algorithm, which will analyze the market basket through the processing of data from barcode scanners, and define goods purchased together. In the sphere, the association mining may be for setting a correct diagnosis. By processing symptom characteristics and various illness factors and defining associations, we may predict disease probability. Moreover, if adding new symptoms in the future, finding relationships between the new data and diseases is possible. healthcare In the public sector, the association mining algorithm can be used for census data processing. As a result, it may help to optimize the planning of public services and businesses such as transport, education, health, setting up new shopping malls, and factories. Unsupervised ML Business Cases Let’s review a couple of real-life business cases where we used unsupervised ML. CLOUD COMPUTING. HARDWARE PERFORMANCE COMPARISON One of our projects involved the usage of unsupervised ML algorithms to compare the performance of virtual machines. Since the project area was a bit tough for understanding, we decided not to use any labels because they might be subjective, but rather build an independent system; thus, we did the following: Created the virtual machine on the cloud Benchmarked it via running different tests in order to measure the performance of VM Collected about 2000 characteristics as raw data Analyzed the gathered raw data and extracted the most valuable benchmarks Compressed benchmarks into several coefficients (parallelization, one core, stability, database, RAM) Calculated the custom coefficient as a balance between the performance and price Selected the best instance type based on their characteristics and price The process described above is a good example of dimensionality reduction technique since we didn’t include all the characteristics to the system but the collapsed information which could be considered as “representative” in terms of results. HUMAN LABELING EVALUATION AND AUTOMATION The project involved the evaluation of human labels, which is now a tough task in data science. We had to find the best ways to evaluate the quality of human labels. Having the data about individuals and legal entities, we should divide it into three classes, analyze it, find specific labels, and predict labeling accuracy. We set the human action label based on specific characteristics, constructed the classifier, and evaluated the quality of these labels. As you can see from the graph above, it’s almost impossible to divide classes without users as a variable because they are too close to each other according to the characteristics and their labels. But let’s take a look at this chart with the data where we included users as a variable, where the final human action labels were dependent on the person. By adding a person into that algorithm, we could achieve the best results. The important thing here is that the data should be independent, which means that it should not have relationships with the person interested in labeling. Thereby, we concluded that there was no proper way to proceed with classification tasks, and it’s better to use raw data. In conclusion, I would say that there is no perfect path for any business case. And unsupervised machine learning is just a tool for getting expected results. It will work well if you are sure that it meets your business requirements. Highlights: Unsupervised machine learning still requires a good quality of data to be processed, although not labeled. A proper data preparation approach drives “right” business insights by improving their accuracy. Unsupervised machine learning algorithms should be chosen depending on a particular business case, not on the popularity of a specific approach and . trends The goal of and is to tackle business problems, but not Data Science ones. Data Science machine learning consulting Written by Liudmyla Taranenko, Data Science Engineer at MobiDev

How To Improve Data Quality When With Unsupervised Machine Learning

Essential Guide to Clustering In Unsupervised Learning

Understanding The Concept of Clustering In Unsupervised Learning

More data we have, better performance we can achieve. However, it is very too luxury to annotate large amount of training data. Therefore, proper data augmentation is useful to boost up your model performance. Authors of (Xie et al., 2019) proposed Unsupervised Data Augmentation (UDA) assistants us to build a better model by leveraging several data augmentation methods. Unsupervised Data Augmentation In natural language processing (NLP) field, it is hard to augmenting text due to high complexity of language. Not every word we can replace it by others such as a, an, the. Also, not every word has synonym. Even changing a word, the context will be totally difference. On the other hand, generating augmented image in computer vision area is relative easier. Even introducing noise or cropping out portion of image, model can still classify the image. Xie et al. conducted several data augmentation experiments on image classification (AutoAugment) and text classification (Back translation and TF-IDF based word replacing). After generating large enough data set of model training, authors noticed that model can easily over-fit. Therefore, they introduce Training Signal Annealing (TSA) to overcome it. Augmentation Strategies This section will introduce three data augmentation in computer vision (CV) and natural language processing (NLP) field. AutoAugment for Image Classification AutoAugment is found by google in 2018. It is a way to augment image automatically. Unlike traditional image augmentation library, AutoAugment is designed to find the best policy to manipulate data automatically. You may visit for model and implementation. here Back translation for Text Classification Back translation is a method to leverage translation system to generate data. Given that we have a model for translating English to Cantonese and visa versa. Augmented data can be retrieved by translating the original data from English to Cantonese and then translating back to English used back-translation method to generate more training data to improve translation model performance. Sennrich et al. (2015) TF-IDF based word replacing for Text Classification Although back translation helps to generate lot of data, there is no guarantee that keywords will be kept after translation. Some keywords carries more information than others and it may be missed after translation. Therefore, Xie et al. uses to tackle this limitation. The concept of TF-IDF is that high frequency may not able to provide much information gain. In another word, rare words contribute more weights to the model. Word importance will be increased if the number of occurrence within same document (i.e. training record). On the other hand, it will be decreased if it occurs in corpus (i.e. other training records). TF-IDF IDF score is calculated by DBPedia corpus. TF-IDF score will be computed for each token and replacing it according to TF-IDF score. Low TF-IDF score will have a high probability to be replaced. If you are interested to use TF-IDF based word replacing for data augmentation, you may visit for python implementation. nlpaug Training Signal Annealing (TSA) After generated large amount of data by using aforementioned skill, Xie et al. noticed  that  model will be over-fitting easily. Therefore, they introduce TSA. During model training, examples with high confidence will be removed from loss function to prevent over-training. The following figure shows the value range of ηt while K is number of categories. If the probability is higher then ηt, it will be removed from loss function. 3 calculation of ηt are considered for different scenarios: Linear-schedule: Growing constantly Log-schedule: Growing faster in the early stage of training Exp-schedule: Growing faster at the end of training. Recommendation The above approach is designed to solve problems which authors are facing in their problem. If you understand your data, you should tailor made augmentation approach it. Remember that golden rule in data science is garbage in garbage out. Like to learn? I am Data Scientist in Bay Area. Focusing on state-of-the-art in Data Science, Artificial Intelligence , especially in NLP and platform related. Feel free to connect with on or . me LinkedIn Github Extension Reading Data Augmentation for Text Unofficial AutoAugment implementation Reference R. Sennrich, B. Haddow and A Birch. . 2015 Improving Neural Machine Translation Models with Monolingual Data E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan and Q. V. Le. . 2018 AutoAugment: Learning Augmentation Strategies from Data Q. Xie, Z. Dai, E Hovy, M. T. Luong and Q. V. Le. . 2019 Unsupervised Data Augmentation

Unsupervised Data Augmentation

How can you apply unsupervised learning on audio data? Unsupervised Learning Unsupervised learning approach is demonstrated by state-of-the-art NLP model (e.g. BERT, GPT-2) to be a good way to learn feature for downstream task. Researchers demonstrated data-driven learned features provide a better audio feature than traditional acoustic feature such as Mel-frequency cepstrum (MFCC). This story will discuss about how can you use unsupervised learning to learn audio feature and applying it to downstream tasks. Unsupervised feature learning for audio classification Lee et al. propose to use convolutional deep belief network (CDBN, aksdeep learning representation nowadays) to replace traditional audio features (e.g. spectrogram and Mel-frequency cepstrum (MFCC)). The original input is spectrogram from each utterance and window size is 20ms with 10ms overlaps. Small window and overlapping setting is common when handling audio input. Believe that computer resource is limited in that time (it was 2009), they leverage principal component analysis (PCA) to reduce dimension before feeding into neural network. The setting for the neural network includes 2 convolutional neural network (CNN) layer with 300 dimensions, filter length of 6 and max-pooling ratio of 3. Visualization of pre-trained layer (Lee et al., 2009) Lee et al. evaluated data-driven feature (CDBN) with traditional features which are spectrogram (i.e. RAW in the following figure) and MFCC for speaker identification, speaker gender classification, phone classification, music genre classification and music artist classification. Test classification accuracy for speak identification by using summary statistics (Lee et al., 2009) Test classification accuracy for speak identification by using all frame (Lee et al., 2009) Test classification accuracy for gender classification (Lee et al., 2009) Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging Xu et al. use spectrogram as raw input to learn vector representation. Asymmetric de-noising auto-encoder (aDAE) is presented in the research paper. The network architecture include encoder (first three layers) and decoder (last three layers) parts. Spectrogram is extracted and feeding into the encoder while the training objective is predicting the middle of frame by using previous and next frames. It is similar to Continuous Bag-of-Words (CBOW) in NLP. CBOW use surroundings to predict the target words. Network architecture of aDAE (Xu et al., 2016) The following model comparisons demonstrate aDAE achieve a better result in general. Model comparison results among labels (Xu et al., 2016) Unsupervised feature learning for audio analysis Meyer et al. also use spectrogram as raw input to learn vector representation. The training object is using previous frame to predict next frame which is similar to language model in NLP. Audio frame predictor (AFP) is presented in this paper. The network architecture include encoder and decoder parts. Spectrogram is extracted with 2.56s sliding window size and 0.64s overlaps and feeding into encoder which includes multiple ConvLSTM layer. ConvLSTM setting use 3x3 filter kernal with ReLu activation and batch normalization function. Model of the audio frame predictor (Meyer et al., 2017) Meyer et al. use two steps procedure to train the data-driven representation. The network is trained by minimizing the mean squared error (MSE) (i.e. encoder to decoder) in the first 6 epochs. In the sixth to ninth epochs, pairwise loss training objected is added to adjust the representation simultaneously. Vector distribution from pre-trained feature (Meyer et al., 2017) Tess accuracy for classification and clustering tasks (Meyer et al., 2017) Like to learn? I am Data Scientist in Bay Area. Focusing on state-of-the-art in Data Science, Artificial Intelligence , especially in NLP and platform related. Feel free to connect with me on LinkedIn or following me on Medium or Github. Reference H. Lee, Y. Largman, P. Pham and Andrew Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks. 2009
Y. Xu, Q.Huang, W. Wang, P.Foster, S. Sigtia, P. J. B. Jackson, and M. D. Plumbley. Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging. 2016
M. Meyer, J. Beutel and L. Thiele. Unsupervised Feature Learning for Audio Analysis. 2017 How can you apply unsupervised learning on audio data? Unsupervised Learning Unsupervised learning approach is demonstrated by state-of-the-art NLP model (e.g. BERT, GPT-2) to be a good way to learn feature for downstream task. Researchers demonstrated data-driven learned features provide a better audio feature than traditional acoustic feature such as Mel-frequency cepstrum (MFCC). This story will discuss about how can you use unsupervised learning to learn audio feature and applying it to downstream tasks. Unsupervised feature learning for audio classification Lee et al. propose to use convolutional deep belief network (CDBN, aksdeep learning representation nowadays) to replace traditional audio features (e.g. spectrogram and Mel-frequency cepstrum (MFCC)). The original input is spectrogram from each utterance and window size is 20ms with 10ms overlaps. Small window and overlapping setting is common when handling audio input. Believe that computer resource is limited in that time (it was 2009), they leverage principal component analysis (PCA) to reduce dimension before feeding into neural network. The setting for the neural network includes 2 convolutional neural network (CNN) layer with 300 dimensions, filter length of 6 and max-pooling ratio of 3. Visualization of pre-trained layer (Lee et al., 2009) Lee et al. evaluated data-driven feature (CDBN) with traditional features which are spectrogram (i.e. RAW in the following figure) and MFCC for speaker identification, speaker gender classification, phone classification, music genre classification and music artist classification. Test classification accuracy for speak identification by using summary statistics (Lee et al., 2009) Test classification accuracy for speak identification by using all frame (Lee et al., 2009) Test classification accuracy for gender classification (Lee et al., 2009) Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging Xu et al. use spectrogram as raw input to learn vector representation. Asymmetric de-noising auto-encoder (aDAE) is presented in the research paper. The network architecture include encoder (first three layers) and decoder (last three layers) parts. Spectrogram is extracted and feeding into the encoder while the training objective is predicting the middle of frame by using previous and next frames. It is similar to Continuous Bag-of-Words (CBOW) in NLP. CBOW use surroundings to predict the target words. Network architecture of aDAE (Xu et al., 2016) The following model comparisons demonstrate aDAE achieve a better result in general. Model comparison results among labels (Xu et al., 2016) Unsupervised feature learning for audio analysis Meyer et al. also use spectrogram as raw input to learn vector representation. The training object is using previous frame to predict next frame which is similar to language model in NLP. Audio frame predictor (AFP) is presented in this paper. The network architecture include encoder and decoder parts. Spectrogram is extracted with 2.56s sliding window size and 0.64s overlaps and feeding into encoder which includes multiple ConvLSTM layer. ConvLSTM setting use 3x3 filter kernal with ReLu activation and batch normalization function. Model of the audio frame predictor (Meyer et al., 2017) Meyer et al. use two steps procedure to train the data-driven representation. The network is trained by minimizing the mean squared error (MSE) (i.e. encoder to decoder) in the first 6 epochs. In the sixth to ninth epochs, pairwise loss training objected is added to adjust the representation simultaneously. Vector distribution from pre-trained feature (Meyer et al., 2017) Tess accuracy for classification and clustering tasks (Meyer et al., 2017) Like to learn? I am Data Scientist in Bay Area. Focusing on state-of-the-art in Data Science, Artificial Intelligence , especially in NLP and platform related. Feel free to connect with me on LinkedIn or following me on Medium or Github . me LinkedIn Medium Github Reference H. Lee, Y. Largman, P. Pham and Andrew Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks. 2009 Y. Xu, Q.Huang, W. Wang, P.Foster, S. Sigtia, P. J. B. Jackson, and M. D. Plumbley. Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging. 2016 M. Meyer, J. Beutel and L. Thiele. Unsupervised Feature Learning for Audio Analysis. 2017 H. Lee, Y. Largman, P. Pham and Andrew Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks . 2009 Unsupervised feature learning for audio classification using convolutional deep belief networks Y. Xu, Q.Huang, W. Wang, P.Foster, S. Sigtia, P. J. B. Jackson, and M. D. Plumbley. Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging . 2016 Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging M. Meyer, J. Beutel and L. Thiele. Unsupervised Feature Learning for Audio Analysis . 2017 Unsupervised Feature Learning for Audio Analysis

How can you apply unsupervised learning on audio data?

Learn how to train and evaluate an unsupervised machine learning model — principal component analysis in this article by Jillur Quddus, a lead technical architect, polyglot software engineer and data scientist. There are numerous real-world use cases, where the number of features available, which may potentially be used to train a model, is very large. A common example is economic data and using its constituents, stock price data, employment data, banking data, industrial data, and housing data together to predict the ( ). Such types of data are said to have high dimensionality. Though they offer numerous features that can be used to model a given use case, high-dimensional datasets increase the computational complexity of machine learning algorithms and more importantly may also result in over fitting. gross domestic product GDP Over fitting is one of the results of the , which formally describes the problem of analyzing data in high-dimensional spaces (which means that the data may contain many attributes, typically hundreds or even thousands of dimensions/features) but where that analysis no longer holds true in a lower-dimensional space. curse of dimensionality Informally, it describes the value of additional dimensions at the cost of model performance. ( ) is an technique used to preprocess and reduce the dimensionality of high-dimensional datasets while preserving the original structure and relationships inherent to the original dataset so that machine learning models can still learn from them and be used to make accurate predictions. Principal component analysis PCA unsupervised Movie recommendation system To better understand PCA, let’s study a movie recommendation use case. Our aim is to build a system that can make personalized movie recommendations to users based on historic user-community movie ratings. The historic user-community movie ratings data that we will use for our case study has been downloaded from GroupLens, a research laboratory based at the University of Minnesota that collects movie ratings and makes them available for public download at . For the purposes of this case study, we have transformed the individual and datasets into a single pivot table where the 300 rows represent 300 different users, and the 3,000 columns represent 3,000 different movies. https://grouplens.org/datasets/movielens/ movies ratings This transformed, pipe-delimited dataset can be found in the GitHub repository at . You can also find the code file for this article at . https://github.com/PacktPublishing/Machine-Learning-with-Apache-Spark-Quick-Start-Guide/tree/master/Chapter05/data/movie-ratings-data and is called movie-ratings-data/user-movie-ratings.csv https://github.com/PacktPublishing/Machine-Learning-with-Apache-Spark-Quick-Start-Guide/blob/master/Chapter05/chp05-02-principal-component-analysis.ipynb A sample of the historic user-community movie ratings dataset that we will study looks as follows: In this case, each movie is a different feature (or dimension) and each different user is a different instance (or observation). This sample table, therefore, represents a dataset containing 5 features. However, our actual dataset contains 3,000 different movies and therefore 3,000 features/dimensions. Furthermore, in a real-life representation, not all users would have rated all the movies, so there will be a significant number of missing values. Such a dataset, and the matrix used to represent it, is described as . These issues would pose a problem for machine learning algorithms, both in terms of computational complexity and the likelihood of over fitting. sparse To solve this problem, take a closer look at the previous sample table. It seems that users that rated Movie #1 highly (Toy Story) generally also rated Movie #2 highly (Monsters Inc.) as well. We could say, for example, that User #1 is of all fans of computer-animated children’s films, and so we could recommend to User #2 the other movies that User #1 has historically rated highly (this type of recommendation system where we use data from other users is called ). representative collaborative filtering At a high level, this is what PCA does — it identifies , called , within a high-dimensional dataset so that the dimensions of the original dataset can be reduced while preserving its underlying structure and still be representative in dimensions! These reduced datasets can then be fed into machine learning models to make predictions as normal, without the fear of any adverse effects from reducing the raw size of the original dataset. Our formal definition of PCA can therefore now be extended so that we can define PCA as the identification of a linear subspace of lower dimensionality where the largest variance in the original dataset is maintained. typical representations principal components lower Returning to our historic user-community movie ratings dataset, instead of eliminating Movie #2 entirely, we could seek to create a new feature that combines Movie #1 and Movie #2 in some manner. Extending this concept, we can create new features where each new feature is based on all the old features, and thereafter order these new features by how well they help us in predicting user movie ratings. Once ordered, we can drop the least important ones, thereby resulting in a reduction in dimensionality. So how does PCA achieve this? It does so by performing the following steps: 1. First, we standardize the original high-dimensional dataset. 2. Next, we take the standardized data and compute a covariance matrix that provides a means to measure how all our features relate to each other. 3. After computing the covariance matrix, we then find its and corresponding . Eigenvectors represent the principal components and provide a means to understand the direction of the data. Corresponding eigenvalues represent how much variance there is in the data in that direction. eigenvectors eigenvalues 4. The eigenvectors are then sorted in descending order based on their corresponding eigenvalues, after which the top eigenvectors are selected representing the most important representations found in the data. k 5. A new matrix is then constructed with these eigenvectors, thereby reducing the original -dimensional dataset into reduced dimensions. k n k Covariance matrix In mathematics, refers to a measure of how spread out a dataset is and is calculated by the sum of the squared distances of each data point, , from the mean , divided by the total number of data points, . This is represented by the following formula: variance xi x-bar N refers to a measure of how strong the correlation between two or more random variables is (in our case, our independent variables) and is calculated for variables and over dimensions: Covariance x y i If the covariance is positive, this implies that the independent variables are positively correlated. If the covariance is negative, this implies that the independent variables are negatively correlated. Finally, a covariance of zero implies that there is no correlation between the independent variables. Here, we are computing the covariance between all variables. A is a symmetric square matrix where the general element ( , ) is the covariance, , between independent variables and (which is the same as the symmetric covariance between and ). Note that the diagonal in a covariance matrix actually represents just the between those elements, by definition. covariance matrix i j cov(i, j) i j j i variance The covariance matrix is shown in the following table: Identity matrix An identity matrix is a square matrix in which all the elements along the main diagonal are 1 and the remaining elements are 0. Identity matrices are important for when we need to find all of the eigenvectors for a matrix. For example, a 3 x 3 identity matrix looks as follows: Eigenvectors and eigenvalues In linear algebra, eigenvectors are a special set of vectors whose remains unchanged when a linear transformation is applied to it, and only changes by a factor. In the context of dimensionality reduction, eigenvectors represent the principal components and provide a means to understand the direction of the data. direction scalar Consider a matrix, , of dimensions ( x ). We can multiply by a vector, (of dimensions x 1 by definition), which results in a new vector, (of dimensions x 1), as follows: A m n A x n b m In other words, However, in some cases, the resulting vector, , is actually a scaled version of the original vector, . We call this scalar factor , in which case the formula above can be rewritten as follows: b x λ We say that is an of matrix , and is an associated with . In the context of dimensionality reduction, eigenvalues represent how much variance there is in the data in that direction. λ eigenvalue A x eigenvector λ In order to find all the eigenvectors for a matrix, we need to solve the following equation for each eigenvalue, where is an identity matrix with the same dimensions as matrix : I A Once all of the eigenvectors for the covariance matrix are found, these are then sorted in descending order by their corresponding eigenvalues. Since eigenvalues represent the amount of variance in the data for that direction, the first eigenvector in the ordered list represents the principal component that captures the most variance in the original variables from the original dataset, and so on. For example, as illustrated in the figure below, if we were to plot a dataset with two dimensions or features, the first eigenvector (which will be the first principal component in order of importance) would represent the direction of most variation between the two features. The second eigenvector (the second principal component in order of importance) would represent the direction of second-most variation between the two features: To help choose the number of principal components, , to select from the top of the ordered list of eigenvectors, we can plot the number of principal components on the axis against the cumulative explained variance on the axis, as illustrated in the next figure, where the explained variance is the ratio between the variance of that principal component and the total variance (that is, the sum of all eigenvalues): k x y Using this as an example, we would select around the first 300 principal components, as these describe the most variation within the data out of the 3,000 in total. Finally, we construct a new matrix by projecting the original dataset into -dimensional space represented by the eigenvectors selected, thereby reducing the dimensionality of the original dataset from 3,000 dimensions to 300 dimensions. This preprocessed and reduced dataset can then be used to train machine learning models as normal. k PCA in Apache Spark Let’s now return to our transformed pipe-delimited user-community movie ratings dataset, , which contains ratings by 300 users covering 3,000 movies. We will develop an application in Apache Spark that seeks to reduce the dimensionality of this dataset while preserving its structure using PCA. To do this, we will go through the following steps: movie-ratings-data/user-movie-ratings.csv 1. First, let’s load the transformed, pipe-delimited user-community movie ratings dataset into a Spark dataframe using the following code. The resulting Spark dataframe will have 300 rows (representing the 300 different users) and 3,001 columns (representing the 3,000 different movies plus the user ID column): user_movie_ratings_df = sqlContext.read

.format('com.databricks.spark.csv').options(header = 'true',

inferschema = 'true', delimiter = '|')

.load('

print((user_movie_ratings_df.count(),

len(user_movie_ratings_df.columns))) 2. We can now generate feature vectors containing 3,000 elements (representing the 3,000 features) using 's , as we have seen before. We can achieve this using the following code: MLlib MLlib VectorAssembler feature_columns = user_movie_ratings_df.columns

feature_columns.remove('userId')

vector_assembler = VectorAssembler(inputCols = feature_columns,

outputCol = 'features')

user_movie_ratings_features_df = vector_assembler

.transform(user_movie_ratings_df)

.select(['userId', 'features']) 3. Before we can reduce the dimensionality of the dataset using PCA, we first need to standardize the features that we described previously. This can be achieved using 's estimator and fitting it to the Spark dataframe containing our feature vectors, as follows: MLlib StandardScaler standardizer = StandardScaler(withMean=True, withStd=True,

inputCol='features', outputCol='std_features')

standardizer_model = standardizer

.fit(user_movie_ratings_features_df)

user_movie_ratings_standardized_features_df =

standardizer_model.transform(user_movie_ratings_features_df) 4. Next, we convert our scaled features into an instance. A is a distributed matrix with no index, where each row is a vector. We achieve this by converting our scaled features data frame into an RDD and mapping each row of the RDD to the corresponding scaled feature vector. We then pass this RDD to 's (as shown in the following code), resulting in a matrix of standardized feature vectors of dimensions 300 x 3,000: MLlibRowMatrix RowMatrix MLlib RowMatrix() scaled_features_rows_rdd =

user_movie_ratings_standardized_features_df

.select("std_features").rdd

scaled_features_matrix = RowMatrix(scaled_features_rows_rdd

.map(lambda x: x[0].tolist())) 5. Now that we have our standardized data in matrix form, we can easily compute the top principal components by invoking the method exposed by 's . We can compute the top 300 principal components as follows: k computePrincipalComponents() MLlib RowMatrix number_principal_components = 300

principal_components = scaled_features_matrix

.computePrincipalComponents(number_principal_components) 6. Now that we have identified the top 300 principal components, we can project the standardized user-community movie ratings data from 3,000 dimensions to a linear subspace of only 300 dimensions while preserving the largest variances from the original dataset. This is achieved by using matrix multiplication and multiplying the matrix containing the standardized data by the matrix containing the top 300 principal components, as follows: projected_matrix = scaled_features_matrix

.multiply(principal_components)

print((projected_matrix.numRows(), projected_matrix.numCols())) The resulting matrix now has dimensions of 300 x 300, confirming the reduction in dimensionality from the original 3,000 to only 300! We can now use this projected matrix and its PCA feature vectors as the input into subsequent machine learning models as normal. 7. Alternatively, we can use 's estimator directly on the dataframe containing our standardized feature vectors to generate a new dataframe with a new column containing the PCA feature vectors, as follows: MLlib PCA() pca = PCA(k=number_principal_components, inputCol="std_features",

outputCol="pca_features")

pca_model = pca.fit(user_movie_ratings_standardized_features_df)

user_movie_ratings_pca_df = pca_model

.transform(user_movie_ratings_standardized_features_df) Again, this new dataframe and its PCA feature vectors can then be used to train subsequent machine learning models as normal. 8. Finally, we can extract the explained variance for each principal component from our PCA model by accessing its attribute as follows: explainedVariance pca_model.explainedVariance The resulting vector (of 300 elements) shows that, in our example, the first eigenvector (and therefore the first principal component) in the ordered list of principal components explains 8.2% of the variance; the second explains 4%, and so on. In this example, we have demonstrated how we can reduce the dimensionality of the user-community movie ratings dataset from 3,000 dimensions to only 300 dimensions while preserving its structure using PCA. The resulting reduced dataset can then be used to train machine learning models as normal, such as a hierarchical clustering model for collaborative filtering. If you found this article interesting, you can explore Machine Learning with Apache Spark Quick Start Guide to combine advanced analytics including machine learning, deep learning neural networks and natural language processing with modern scalable technologies including apache spark to derive actionable insights from big data in real-time. Machine Learning with Apache Spark Quick Start Guide can help you become familiar with advanced techniques for processing a large volume of data by applying machine learning algorithms.

Principal Component Analysis — Unsupervised Learning Model

Unsupervised learning may sound like a fancy way to say “ but it’s actually a pattern-finding technique for mining inspiration from your data. It has nothing to do with machines running around without adult supervision, forming their own opinions about things. Let’s demystify! let the kids learn on their own not to touch the hot oven” This post is beginner-friendly, but assumes you’re familiar with : the story so far Machine learning is all about . labeling things using examples If you train your system by feeding it the answers you’re looking for, you’re doing . supervised learning To with supervised learning you need to know what labels you want. (Not so with unsupervised.) get started Standard jargon includes . instance, feature, label, model, and algorithm What’s unsupervised learning? Your mission? Put these six images into two groups however you like. Check out the six above. What’s missing? These photographs are not accompanied by labels. No worries, your brain is pretty good at unsupervised learning. Let’s try it. instances Think about how you’d like to split these images into two groups. There are no wrong answers. Ready? Clustering the data In a live class, Googlers shout out answers like “ ” “ ” “ ” and so on. Let’s examine the first answer. sitting versus standing, can see a wooden floor versus can’t, cat selfie vs not cat selfie, One way to split the images in to two clusters: sitting versus standing. Well, “sitting” versus standing. Unsupervised learning‘s secret labels If you chose to define your clusters based on whether the cats are standing, what are the labels your system outputs? Machine learning is about , after all. labeling things If you’re thinking “ ” are the labels, think again! That’s the recipe (model) you’re using for creating your clusters. The labels in unsupervised learning are far more boring: something like “ ” or “ ” or “ ”. They simply indicate group membership, and they have no additional human-interpretable (or poetic) meaning. sitting vs standing Group 1 and Group 2 A or B 0 or 1 Unsupervised learning’s labels simply indicate cluster membership. They have no higher human-interpretable meaning, as disappointingly boring as that may feel. All that is happening here is that the algorithm groups things by similarity. The similarity measure is specified by the choice of algorithm, but why not try as many as possible? After all, you don’t know what you’re looking for and that’s okay. Think of unsupervised learning as a sort of mathematical version of making “ ” birds of a feather flock together. Like a , the results are there to help you dream. Don’t take whatever you see in them too seriously. Rorschach card Look again! As the proud mother of these two individual cats, I’m saddened that in the 50 or so times I’ve taught this lesson, only one audience noticed: “ ” Instead it’s answers like “ ” or “ ” or sometimes even “ ” (Awww.) Cat 1 versus Cat 2. sitting, standing wooden floor absent/present ugly cats versus pretty cats. Turns out these were photos of my two individual cats! Maybe you spotted it, but most of my audiences don’t… unless I give them the labels (supervise their learning). If I’d presented the data with name labels in the first place and then asked you to classify the next photo, I bet you’d find the task easy. Lessons learned Imagine I’m a novice data scientist getting started with unsupervised learning and (naturally!) interested in my own two cats. I won’t be able to unsee my cats when I look at these data. Because my cuddlebugs are so meaningful to me, I expect my unsupervised machine learning system to be able to recover the only thing worth caring about here. Oops! Before this decade, computers couldn’t even hope to compete with the bestpattern finder in the world for this kind of task: the human brain. This is easy for people! So why did the thousands of Googlers who saw these unlabeled photos miss the “ ” answer? Cat 1 versus Cat 2 Think of unsupervised learning as a sort of mathematical version of making “birds of a feather flock together.” Just because something’s interesting to me doesn’t mean my pattern finder will find it. Even if the pattern finder is awesome, I didn’t tell it what I’m looking for, so why would I expect my to deliver it? This isn’t magic! If I don’t tell it what the right answers are... I get what I get and I don’t get upset. All I can do is look at the clusters the system returns for me and see if I find them inspiring. If I don’t like 'em, I just run a different unsupervised algorithm (“Someone else in the audience, split them for me a different way”) over and over until something feels interesting. learning algorithm The results are a Rorschach card to help you dream. There’s no guarantee that anything inspiring will come out of the process, but it doesn’t hurt to try. Exploring the unknown is supposed to be a bit of an adventure, after all. Have fun with it! In the , we’ll look at a cautionary tale of what can go wrong if you forget that the labels are just an inspiration and shouldn’t be taken too seriously, let alone treated as human-interpretable. They’re just there to give you ideas about what you might like to dive into next. next episode If you’re feeling artsy, you might also enjoy my follow up article , which describes how to use unsupervised learning and three other popular techniques in AI art. Using AI as a Perception-Altering Drug Unsupervised learning helps you find inspiration in data by grouping similar things together for you. There are many different ways of defining similarity, so keep trying algorithms and settings until a cool pattern catches your eye. Summary:

Stories, News & Mentions about UNSUPERVISED

Unsupervised’s Stories on HackerNoon