When is a Model’s Training Error Calculated?

Neural Networks are typically trained by adjusting their weights to lower an error function.  This same process also applies to other models, such as Support Vector Machines.  There are a wide variety of ways to train a neural network.  Most of these trainers report the overall error of the neural network and training data.  The error is typically the difference between the expected output of a model, and the ideal expected output for a trained model.  These ideal output values are always included in a supervised training set. The overall training error is the average error, across all training elements, for a single iteration of training.

Continue reading

Mapping Data to a Machine Learning Model’s Inputs and Outputs

How many inputs does my model have?  How many outputs does my model have?  These two related questions can often lead to great confusion when setting up a model such as a neural network or a support vector machine.  These models work by accepting a fixed number of inputs and returning a fixed number of outputs based on those inputs.  If you need to review how models work this article may help.

Ideally your model would have the same number of inputs and outputs as your collected data. However, this is rarely the case.  This is because data are rarely presented to the model in exactly the same way as you originally received the data.

Continue reading

So you Want to be a Data Scientist?

Harvard business review calls it the sexiest job of the 21st century. But, what skills are needed to become a data scientist, and how can you get these skills?  I began as an advanced computer programmer with business knowledge.  Open Source involvement in Artificial Intelligence gave me the foundation to  move into a data science role.

There are really three critical skills that a data scientist must posses. A data scientist must be a statistician, domain expert and hacker – not necessarily in that order. There are different types of data scientist. Each type will be stronger in one of these three skills. Lets take a look at each of these three skills and see how you might build up your knowledge.

Continue reading

Press Release for Artificial Intelligence for Humans Volume 2: Nature Inspired Algorithms


February 20, 2014

Jeff Heaton,  jheaton at heatonresearch.com

Nature-inspired algorithm book teaches programmers basics of Artificial Intelligence

Second volume of the popular Artificial Intelligence for Humans series launches on Kickstarter

St. Louis, MO – The birds and the bees serve as the muse behind the mathematical formulas featured in the latest book from data scientist Jeff Heaton that releases today on Kickstarter.  This new volume of the Artificial Intelligence for Humans series introduces algorithms inspired by elements of nature to teach programmers the fundamentals of AI.

“Artificial Intelligence for Humans is a series of books that presents the topic in a mathematically gentle manner,” said Heaton.  “Computer programmers are not necessarily wizards of all the Calculus, Linear Algebra and Statistical concepts that are required to work with AI.  This series will help programmers apply the ideas of AI to data analysis by fully explaining all the relevant math techniques and providing real-life examples.”

As an important component to the fields of Data Science and Big Data, Artificial Intelligence allows businesses to capitalize on vast amounts of collected data so they can tailor their products to customer needs.   Personalizing products for customers through data mining offers businesses the ability to enhance their services and profitability.

Heaton’s latest volume on AI explores how genomes, cells, ants, birds, and evolution as well as other natural processes influence programming and provides useful applications for the IT professional interested in delving into this dynamic field of computer science.

Programming examples are provided in Java, C# and Python.  Additional languages may be added as stretch goals during the Kickstarter campaign.

Heaton will seek Kickstarter pledges to support this book, prior to its August 2014 publication date, at levels between $5 and $250.

The first volume, Fundamental Algorithms (ISBN: 978-1493682225), attracted 818 backers and achieved 755% of its Kickstarter funding goal on July 10, 2013.  It was delivered on time to project supporters in December 2013.

About Jeff Heaton: Data Scientist, computer programmer and indie publisher specializing in Artificial Intelligence, Jeff is an active technology blogger, open source contributor, and author of more than ten books. Having worked fifteen years in the life insurance industry, Jeff is a Fellow of the Life Management Institute and a senior member of the IEEE.

For more information about the Artificial Intelligence for Humans series, please contact Jeff Heaton (jheaton@heatonresearch.com or visit the following sites:

Artificial Intelligence for Humans, Volume 2: Nature Inspired Algorithms

Artificial Intelligence for Humans, Volume 2: Nature Inspired Algorithms


Overfitting happens to humans too

Does over fitting happen to human beings? Or, is this a phenomenon associated only with machine learning? Just a random thought that I had today.

Over fitting is a common problem that most Artificial Intelligence practitioners face. Wikipedia defines over fitting as “overfitting occurs when a statistical model describes random error or noise instead of the underlying relationship”.  Basically, the model becomes fixated on outliers.  This is often the result of overtraining (obsession) on a relatively small data set.

Does the human brain do this too?  Of course it does!  Stereotypes are essentially the “human brain” equivalent of over fitting.  Maybe your parents told that everyone from the Republic of Elbonia, acts in a particular way.  (Potential outlier datapoint!)  You then watch a movie that also portrays this peculiar trait of Elbonians. (Another potential outlier!)  Maybe your parents saw that same movie!  Danger, we are now counting the same outliers multiple times and now counting dependent variables amount the independent variables they are drawing from!

However, if we actually sample 1,000 Elbonians vs 1,000 members of the general population, we find out that the occurrence of said statistical trait no significant statistical variance.  Said human’s perception of Elbonians is overfit.



iOS App Development for Teens by Teens on Kickstarter

I just backed iOS App Development for Teens by Teens on Kickstarter.  This project seeks to produce an iOS programming book targeted at teens.  That fact alone is very interesting, however, what makes this project unique, is that the author is a teen himself.  Chip Beck is a fifteen year old high school student at Hickory High School in the small town of Hermitage, PA. He also has a number of interesting, and well reviewed, apps on the App Store.

There are several things that I like about this book project.  First, Chip has already backed 35 Kickstarter projects over the last few years.  That is both impressive and commendable.  He has been a valuable member of the Kickstarter community before striking out on his own.  Secondly, he is focusing on ObjectiveC.  There are several more “virtual” approaches to iOS app development.  However, ObjectiveC is really “the way” to develop software for the Mac.  Third, he has experience, as he has been developing apps for a number of years.  His parents also own a printing company, and will surely be able to give their son some guidance in his future publishing endeavors.

This project has already attracted some media attention.  A local television staten interviewed the teen about this project.  Additionally, the LA Croudfunding Examiner posted an article as well. Chip is only asking for $1,000 for his project.  I wish Chip the best of luck.  I launched my last book through through Kickstarter last summer, so I know how much work this can be.  You can find his project at the following URL.


Fulfilling a Book/EBook Kickstarter

Last summer I ran a Kickstarter project from June 10, 2013 to July 10, 2013 for my new book Artificial Intelligence for Humans, Volume 1: Fundamental Algorithms. The book is done, and I’ve just finished sending off several hundred physical books around the globe. For this post I will explain how I actually handled the fulfillment of all of these orders. I also posted previously on the actual Kickstarter campaign itself.

Continue reading