Training the Machines Update

Thanks for the great work helping us find fruits, flowers and unfolded leaves on Acer (the maples). That is going gangbusters and we hope you can continue the great pace of effort!

We wanted to pass along some good news about how well you did on the first training the machines expedition which focused on Prunus (cherries and allies). The results below are organized by “trait” and includes the total classifications, correct classifications and percentages. We also provide the true positives and true negatives as well and use that calculate what is called “sensitivity” and “specificity.” Sensitivity is a measure of the correct positive against the total true positives. And specificity is the same idea for true negatives. Accuracy is the overall ratio of correct classifications to total classifications from the gold standard. All of our assessments treat a hand-coded dataset that we created at Notes from Nature as a gold standard for comparison.
13691529605_073b9cf02d_k
Whenever we do these comparisons, we go back to our hand-coded dataset and check our hand-coded results against what appear to be mistakes. As it turns out, we had a number of mistakes in our “gold standard” that you helped us find. That is, the numbers are even better than what is reported below. You guys helped us find problems in our original “gold standard” scoring! That’s super important when developing supervised machine learning datasets. Overall, these are extremely encouraging results! We will be providing some further details about this work in follow up post but, again, this is very exciting news!
Unfolded leaves:

Overall:
Total classifications: 2998
Correct classifications: 2935
Accuracy: 97.9%

Total true positives: 2806 (0.9359573048699132)
Correct positives: 2750
Sensitivity: 98%
Total true negatives: 192 (0.06404269513008673)
Correct negatives: 185
Specificity: 96.35%

Flowers:

Overall:
Total classifications: 2998
Correct classifications: 2918
Accuracy: 97.33%

Total true positives: 1592 (0.5310206804536357)
Correct positives: 1545
Sensitivity: 97.05%

Total true negatives: 1406 (0.46897931954636424)
Correct negatives: 1373
Specificity: 97.65%

Fruits:

Overall:
Total classifications: 2998
Correct classifications: 2904
Accuracy: 96.86%

Total true positives: 796 (0.26551034022681785)
Correct positives: 739
Sensitivity: 92.84%

Total true negatives: 2202 (0.7344896597731821)
Correct negatives: 2165
Specificity: 98.32%

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: