What is AI Training Data?

Article by Daniel Smith | December 28, 2019

Training data is absolutely essential to the development of any machine learning model. The data you use will define your project, so a clear understanding of how it works will drastically improve your chances of success. We’ve briefly discussed AI training data on our website, but it’s worth explaining in a little more detail. Let’s dive into the world of training data and figure out why it’s so important.

 

What is training data?

Essentially, training data is the textbook that will teach your AI to do its assigned task, and will be used over and over again to fine-tune its predictions and improve its success rate. Your AI will use training data in several different ways, all with the aim of improving the accuracy of its predictions. It does this through the variables contained in the data. By identifying these and evaluating their impact on the algorithm, data scientists can strengthen their AI through copious adjustments. The best data will be extremely rich in detail, capable of improving your AI after hundreds of training cycles by hinting at a wide range of variables that affect your algorithm.

The majority of training data will contain pairs of input information and corresponding labeled answers, which are sometimes called the target. In some fields, it will also have highly relevant tags, which will help your AI to make more accurate predictions. However, since variables and relevant details are so important in the training process, datasets for different machine learning tasks will often look very different from each other. For example:

 

  • In sentiment analysis, training data is usually composed of sentences, reviews or tweets as the input, with the label indicating whether that piece of text is positive or negative.
InputLabel
The coffee here is great!Positive sentiment
Not a fan of the cake, though.Negative sentiment

 

  • In image recognition, the input would be the image, while the label suggests what is contained within the image.
InputLabel
Dog
Cat

 

  • In spam detection, the input is an email or text message, while the label would provide information about whether the message is spam or not spam.
InputLabel
Hi all – just writing to confirm that the meeting will be held at 12.00.Not spam
Tokyo grandmas are making 2 million yen a month from this CRAZY scheme!!Spam

 

  • In text categorization, sentences provide the input while the target would suggest the topic of the sentence, such as finance or law.
InputLabel
Despite an early red card, the champions were two goals up at half-time.Sports
If a new agreement is concluded between lessor and lessee, the terms of this contract shall be considered null and void.Legal

 

These are, of course, quite basic examples. It’s also possible to have multiple labels for one piece of raw input data, particularly if this data takes the form of long pieces of text like paragraphs, online comments, or even articles. In text categorization, for example, it’s possible to label each piece of data with multiple categories, depending on the classification system the annotator is using. For entity extraction, one paragraph could be annotated with several labels on the phrase or word level, in order to provide more information about the semantic meaning of phrases and the relationships between different sections of the text.

Looking at this, it quickly becomes clear that relevance and detail are a crucial element of good training data. If two different AI programs use the same training data, it will result in at least one crippled model. This is true even if both programs deal with the same broad category of input information, such as sentences. With this in mind, you can start thinking about the specific types of data and tags that you’ll need in order to provide your model with the best possible training.

 

The 3 types of data for machine learning

Despite the fact that most training data is straightforward in its composition, it’s not used as one homogenous mass. In fact, training is complex and involves several interlocking processes, all of which our dataset has to serve. There are three types of training data necessary to build a machine learning model, with each performing a different role.

Before going any further, it’s worth noting that the term ‘training data’ has two separate meanings. Just to complicate things, training data is not only used as an umbrella term for the total data needed for your project, but also refers to one of these specific subsets of data. This initially sounds confusing. However, the three types of data differ in several important ways. Once you’re aware of this, it should be easy to understand exactly what your data scientists are referring to.

  • Training data is the part of your data which you use to help your machine learning model make predictions. Your model will be run on this set of data exhaustively, churning out results which your data scientists can use to develop your algorithm. It’s the largest part of your overall dataset, comprising around 70-80% of your total data used in the project.
  • Validation data is a second set of data, also containing input and target information, which the machine learning model has never seen before. By running the model on validation data, it’s possible to see whether it can correctly identify relevant new examples. This is where it’s possible to discover new values that are impacting the process. Another common problem often identified during validation is overfitting, where the AI has been wrongly trained to identify examples that are too specific to the training data. As you can imagine, after validation, data scientists will often go back to the training data and run through it again, tweaking values and hyperparameters to make the model more accurate.
  • Testing data comes into play after a lot of improvement and validation. While validation data has tags and target information left on as training wheels, testing data provides no help to the model at all. Asking the model to make predictions based on this data is meant to test whether it will work in the real world, where it won’t have helpful tags scattered around. The final test is the moment of truth for the model, to see if all the hard work has paid off.

It’s important to note that these three types of data work best together if they are all smaller parts of one, overarching dataset. This helps to ensure that all examples are consistent and relevant to the goals of the project. The complete pool of data should be both split and ordered into these 3 categories randomly to avoid selection bias.

 

Why is training data important?

Quite simply, without training data there is no AI. The cleanliness, relevance and quality of your data has a direct impact on whether your AI will achieve its goals. It’s best to think of training data in parallel with a human example of learning. Give a student an outdated textbook with half the pages missing and they won’t come close to passing their course. Similarly, without quality data, your AI will learn to do its job haphazardly, if at all. In the same way that you want to throw the weight of world-renowned professors behind your star pupil, your AI deserves the best data you can find, bursting with detailed tags and relevant annotations. Only then will your AI project catapult your business into the next stage of development.

 

How much training data do you need?

It’s almost impossible to figure out the exact number of data points you’ll need to train your model. There are a wide range of factors that each have a different degree of influence on the size of your dataset. In fact, trying to calculate the amount of training data you need would probably require another machine learning model, dedicated to figuring out the exact relationship between these influences.

Some of the main factors that affect your training data needs are as follows:

  • Complexity of model: Every model has a number of parameters to consider in order to go about its task. For models with clearly defined tasks, this is a relatively small number. More complex models can have much broader tasks without clearly defined limits, meaning they have a much larger range of parameters to consider. Every extra parameter your model has to account for will increase the amount of training data you need.
  • Training method: Many traditional machine learning algorithms are trained using structured learning. This method quickly reaches a point where additional data has very little ROI. However, more complex models rely on deep learning, which allows them to figure out their own parameters and improve independently. These models require a lot more data and also have a longer learning curve where extra data will positively impact results. As a result, your chosen training method will have a big effect on how much training data is necessary for your project.
  • Labeling needs: Different machine learning projects require different types of data. Some types of training data can support multiple labels or annotations, resulting in a need for fewer actual samples. For example, 1,000 sentences of input data for sentiment annotation will only result in one label per sentence – positive, neutral, or negative. However, 1,000 sentences for named entity recognition may yield four or five labels per sentence. This may mean that you need to collect fewer overall sentences. How you intend to use your data is therefore crucial to figuring out how much training data you need to collect.

It’s probably impossible to figure out the exact number of data points needed for any given machine learning project. That said, there are ways to estimate the general amount of training data you’ll need. Two of the more common ways of doing this are:

  • Rule of 10: This is a common way to quickly estimate how much data a model might need – although it’s not without drawbacks. Simply put, a model will usually need ten times more data than it has degrees of freedom. A degree of freedom could be a parameter, an attribute of a data point, or even just a column in your dataset. For complex models this rule is difficult to apply, but it’s useful at times when you need a rough estimate to keep everything moving.
  • Learning Curves: For a deeper dive into this problem, you could consider plotting your results on a graph to figure out the relationship between dataset size and the ability of your model. By doing this, you should be able to identify the point where more data provides diminishing returns. Although this requires you to create a few logistic regression problems, it provides you with a more accurate result than the rule of 10.

In most cases, it’s best to simply begin working with the data you have and add more once the need for it becomes obvious. However, data quantity is an important issue that shouldn’t be dismissed just because it’s difficult to solve. For more information and some concrete examples, check out our in-depth article on how much training data you need.

 

Where can I find training data?

Finding relevant training data can be a surprisingly difficult process. Datasets are usually created for either one unique model or for general use – neither of which really suits your project. However, this can also work in your favor. If you go through the effort of finding highly relevant data and tagging it accurately, you’re saving yourself weeks of time further down the line – and there’s a good chance you’re already one step ahead of the competition.

Below are some popular places to search for machine learning training data:

  • Public sources: The machine learning community has developed a few interesting repositories that occasionally contain helpful data. However, it’s important to bear in mind that these datasets usually aren’t built for your specific purpose. The data here might not be clean or share any similarities with the rest of your dataset. That said, it’s worth taking a look just in case. Kaggle is a good place to start.
  • Crowdsourcing: There are a range of crowdsourcing sites who can provide you with large quantities of cheap, customized data. However, assessing the quality is up to you. The crowd workers will be largely relying on you for a clear understanding of what is required and may not have any knowledge of what useful training data looks like. If you’re prepared for the required level of management, crowdsourcing can be a powerful tool to leverage in your pursuit of a great dataset.
  • Training data providers: A step above more general crowdsourcing websites, some specialist providers use large workforces to create and annotate training data. Since they’re experienced with a vast range of project guidelines, these providers can usually customize their workflows to suit you. Whether you’re starting from scratch or already have some pieces in place, these providers should be able to adapt to your specific needs.

Wherever you choose to source your data, make sure that your provider has both extensive quality processes and the flexibility to take your needs into account. This will help you to ensure that your training data is both ready for use and relevant to your model. Take the time to check their results and be ready to work with them to improve the process. After all, a reliable provider of training data is one of your most valuable allies in machine learning. Choose wisely and you’ll have a trusted partner that you can return to again and again as your model develops.

 

Custom training data

Training data is a crucial piece of your machine learning setup. A proper focus on data collection, cleansing, and annotation will give your engineers and data scientists the best tool for the job. Invest in custom data and you’ll see your team repay you tenfold down the line.

While you’re searching for that perfect partner, it’s worth considering Lionbridge for all your training data needs. From collection to annotation and validation, we’ve been helping people like you to improve their training data for the last 20 years. We offer a choice between an involved approach that uses the full range of our expert project managers and annotators, or keeping full control of your data by using our annotation tool – and everything in between.

Read more about our data annotation services or contact us today for a free trial to see what we’re capable of.

Get in touch for a free evaluation
The Author
Daniel Smith

Daniel writes a variety of content for Lionbridge’s website as part of the marketing team. Born and raised in the UK, he first came to Japan by chance in 2013 and is continually surprised that no one has thrown him out yet. Outside of Lionbridge, he loves to travel, take photos and listen to music that his neighbors really, really hate.

Welcome!

Sign up to our newsletter for fresh developments from the world of training data. Lionbridge brings you interviews with industry experts, dataset collections and more.