Problems With The Data
The LabelMe dataset has some problems that should be noted. Some are inherent in the data, such as the objects in the images not being uniformly distributed with respect to size and image location. This is due to the images being primarily taken by humans who tend to focus the camera on interesting objects in a scene. However, cropping and rescaling the images randomly can simulate a uniform distribution. Other problems are caused by the amount of freedom given to the users of the annotation tool. Some problems that arise are:
- The user can choose which objects in the scene to outline. Should an occluded person be labeled? Should the sky be labeled?
- The user has to describe the shape of the object themselves by outlining a polygon. Should the fingers of a hand on a person be outlined with detail? How much precision must be used when outlining objects?
- The user chooses what text to enter as the label for the object. Should the label be person, man, or pedestrian?
The creators of LabelMe decided to leave these decisions up to the annotator. The reason for this is that they believe people will tend to annotate the images according to what they think is the natural labeling of the images. This also provides some variability in the data, which can help researchers tune their algorithms to account for this variability.
Read more about this topic: Label Me
Famous quotes containing the words problems and/or data:
“It’s so easy during those first few months to think that the problems will never end. You feel as if your son will never sleep through the night, will always spit up food after eating, and will never learn to smile—even though you don’t know any adults or even older children who still act this way.”
—Lawrence Kutner (20th century)
“Mental health data from the 1950’s on middle-aged women showed them to be a particularly distressed group, vulnerable to depression and feelings of uselessness. This isn’t surprising. If society tells you that your main role is to be attractive to men and you are getting crow’s feet, and to be a mother to children and yours are leaving home, no wonder you are distressed.”
—Grace Baruch (20th century)