Machine Learning
Text Classification
New way of communicating your wishes to computer!!
What is text classification??
Text classification is one of the important and typical task in supervised machine learning (ML).
It is the process of assigning a label or category to particular sentence, document or any kind of textual data.
It’s one of the fundamental tasks in Natural Language Processing (NLP) with broad applications such as sentiment analysis, spam detection, emotion detection and intent detection.
Why text classification??
Every day tons of data is generating in the form of emails, web pages, social media, surveys, etc. This text can be extremely rich source of information, But most of this data is unstructured and therefore extracting insights from it can be hard and time-consuming.Here comes the idea of using Text classification.
Text classifiers can be used to organize, structure, and categorize pretty much anything. For example, reports or news articles can be organized by topics, brand mentioned can be categorized by sentiment, and so on.
Approaches of Automatic Text Classification
There are many approaches to automate text classification, which can be grouped into three different types of systems:
(1) Rule-based systems - Classify text into categories based on some set of handcrafted linguistic rules.
(2) Machine Learning based systems - Classify text based on ML model which is trained on some pre-labeled examples as training data.
(3) Hybrid systems - It is the combination of above 2 approaches.
General steps for ML based text classification
Step 1: Gather Data -
Gathering data is the most important step in solving any ML problems as classifier can only be as good as the dataset it is built from. More the data, we can built more generalize model.
Step 2: Explore Your Data -
Understand features of data before building a model can help in creating better model.This includes Loading of data, Check for number of classes, number of samples, imbalance of data, etc.
Step 3: Prepare Your Data -
Before data can be fed to a model, it needs to be transformed to a format the model can understand. This includes steps like tokenization, cleaning of data, converting data to numeric form using TF-IDF, countvectorizer, etc.
Step 4: Build, Train, and Evaluate Your Model -
Find appropriate ML algorithm to build the model based on data features which can convert tokens of text sequences & thereby construct the model architecture.
Training of model involves making a prediction based on the current state of the model, calculating how incorrect the prediction is, and updating the weights or parameters of the network to minimize this error and make the model predict better.
Evaluating model involves measuring the performance of model.
Step 5: Tune Hyperparameters -
Tuning hyperparameters will help refine model to better represent the particularities of the problem at hand.Some of the important hyperparameter are Number of layers in the model, Number of units per layer, Dropout rate, Learning rate, etc.Play around with these hyperparameters and see what works best.
Step 6: Deploy Your Model -
You can deploy the model using Google platform, Azure platform or else using Django Rest Framework or Flask .
Applications of Text Classification
Spam Classification
Spam classifier can tell whether a given mail is spam or not!
News Article Classification
Some of my exciting Machine Learning Blog!!
Machine Learning Blog
Natural Language Processing Basics
