Label class & attributes

ML Insights
#1

Hello All,

In another topic of the forum, I asked information about the best practices in the annotation work.

My question was:

“In terms of best practices related to the annotation work, with the focus to optimise the ML training, should I make as many labels as there are variations of one sort of object (i.e. analogical clock, digital clock, Grandfather clock…) or is it fine for the ML training to make just one label to cover all the variations of the object. (i.e. Clock)”

treebeard kindly replied:

" What we saw often is that this is where attributes are very useful.
You want to be careful that you don’t unfairly penalise the neural network (NN) and also that there is good representation for different events.

For example:
If you make a different label class for every brand of Sedan car then the NN will get penalised when guessing BMW vs Mercedes and waste a lot of energy trying to resolve that.
There it makes more sense to have class as “sedan” and assign an attribute as brand that can focus a different model entirely on trying to make that distinction.

To your example, it might make sense to have class a clock and then attributes as analog, digital etc."

Let’s imagine that the annotation work is completed, I have labels of clocks containing attributes to describe the various types of clock : “analogic, digital, grandfather_clock”

My question to you is this one: are you aware of a method (via opencv, python, yolov3, other?) to retrieve the type attribute of the clock that you identify in a picture that you are scanning.
(my current work environment is Python 3.7 & Opencv to open,read,identify objects in a picture)

Up to now, I was unfortunate in my various investigations on Internet.

Thanks for the attention you’ll grant to my question!

TM

#2

Hey Thierry,

The question seems unclear to me. I take it you mean that after you’ve completed doing annotations, you would like to use this training data to train a neural network that can predict the clock attributes given a clock image. I’m going to answer this question, if this is not your question, please feel free to elaborate.

You can start out by simply training a classifier. Since you have one attribute (clock type) with 3 attribute values (analog, digital, grandfather), you will have 3 classes. Train the network by feeding in images of clocks, and predicting one of the 3 classes.
If you want to do use more advanced classification techniques, you can look at classifiers that leverage graph neural networks to enhance attribute prediction (since presence of one attribute can also indicate the presence of another).

If you would like to detect or segment clocks in an image and predict their types, you can look at models like Faster/Mask-RCNN and use the same 3 classes for training, along with the bbox or mask annotations to find the clock in the image and predict it’s type together.

Hope this helped!

#3

Hello Hasnain,

Many thanks for your reply.

Your understanding of my question : “you would like to use this training data to train a neural network that can predict the clock attributes given a clock image. I’m going to answer this question” is correct !

Your answer is what I need!

Many thanks again!