Problems With The Data
The LabelMe dataset has some problems that should be noted. Some are inherent in the data, such as the objects in the images not being uniformly distributed with respect to size and image location. This is due to the images being primarily taken by humans who tend to focus the camera on interesting objects in a scene. However, cropping and rescaling the images randomly can simulate a uniform distribution. Other problems are caused by the amount of freedom given to the users of the annotation tool. Some problems that arise are:
- The user can choose which objects in the scene to outline. Should an occluded person be labeled? Should the sky be labeled?
- The user has to describe the shape of the object themselves by outlining a polygon. Should the fingers of a hand on a person be outlined with detail? How much precision must be used when outlining objects?
- The user chooses what text to enter as the label for the object. Should the label be person, man, or pedestrian?
The creators of LabelMe decided to leave these decisions up to the annotator. The reason for this is that they believe people will tend to annotate the images according to what they think is the natural labeling of the images. This also provides some variability in the data, which can help researchers tune their algorithms to account for this variability.
Read more about this topic: Label Me
Famous quotes containing the words problems with, problems and/or data:
“I am always glad to think that my education was, for the most part, informal, and had not the slightest reference to a future business career. It left me free and untrammeled to approach my business problems without the limiting influence of specific training.”
—Alice Foote MacDougall (1867–1945)
“I’m no good at being noble, but it doesn’t take much to see that the problems of three little people don’t amount to a hill of beans in this crazy world. Someday you’ll understand that.”
—Julius J. Epstein, U.S. screenwriter, Philip Epstein, screenwriter, Howard Koch, screenwriter, and Michael Curtiz. Rick Blaine (Humphrey Bogart)
“Mental health data from the 1950’s on middle-aged women showed them to be a particularly distressed group, vulnerable to depression and feelings of uselessness. This isn’t surprising. If society tells you that your main role is to be attractive to men and you are getting crow’s feet, and to be a mother to children and yours are leaving home, no wonder you are distressed.”
—Grace Baruch (20th century)