Author Topic: Composition Aware Search tool announced (Read 5154 times)

Jo Ann Snover · « **on:** October 12, 2017, 11:13 »

https://www.theverge.com/2017/10/12/16463768/shutterstock-machine-learning-image-search-composition

https://venturebeat.com/2017/10/12/shutterstocks-new-visual-search-tool-lets-you-search-for-images-by-composition/

Press release here

https://www.shutterstock.com/press/15118

Blog

https://www.shutterstock.com/blog/composition-aware-search-tool

Just playing around with it a bit suggests that it could grow into something interesting, although it might need some rethinking about what search input is. For example, if I wanted an image with a woman on both the left and right of the image and copy space in the center, I can't use the little box where I move around tokens to find that as I only get one "woman" token for the single search term.

I'm sure buyers will make suggestions, so it'll be interesting to see how SS enhances this tool

Jo Ann Snover · « **Reply #1 on:** October 12, 2017, 11:22 »

I found some interesting "spam" the new search introduced that had nothing to do with titles, descriptions or keywords in the images being wrong.

I typed holly for the keyword and added the "copy space" token. A number of the items returned in the search had Christmas greenery but no holly anywhere. I checked on keywords for a couple of them and they were clean.

Perhaps the object identification algorithm finds associations within the collection - lots of holly images have pine cones and fir branches in them so lets return some of those images in the search results as well?

Same sort of thing with a search for Aruba with that token at the bottom center and copy space token top center (to get images with lots of blue sky up top for copy space). The arrangement of elements was OK, but the images were from Bali, Mexico, St. Martin, Tanzania (and the ones I checked were all correctly keyworded). That might be OK if I had typed beach for a keyword, but I typed a specific location and that was essentially ignored.

We need less spam, not more

Pauws99 · « **Reply #2 on:** October 12, 2017, 12:00 »

Quote from: Jo Ann Snover on October 12, 2017, 11:22

I found some interesting "spam" the new search introduced that had nothing to do with titles, descriptions or keywords in the images being wrong.

I typed holly for the keyword and added the "copy space" token. A number of the items returned in the search had Christmas greenery but no holly anywhere. I checked on keywords for a couple of them and they were clean.

Perhaps the object identification algorithm finds associations within the collection - lots of holly images have pine cones and fir branches in them so lets return some of those images in the search results as well?

Same sort of thing with a search for Aruba with that token at the bottom center and copy space token top center (to get images with lots of blue sky up top for copy space). The arrangement of elements was OK, but the images were from Bali, Mexico, St. Martin, Tanzania (and the ones I checked were all correctly keyworded). That might be OK if I had typed beach for a keyword, but I typed a specific location and that was essentially ignored.

We need less spam, not more

I did my trusty aardvark test.......not an aardvark to be seen....more work needed but an interesting concept.

kuriouskat · « **Reply #3 on:** October 12, 2017, 13:02 »

Interesting, but didn't really return what I asked for, which was tomato on left, cucumber on right and text space in the middle. That didn't feel like a complex request, and some of the returns did fit the bill, however most didn't. From the first 12 images, 6 didn't have cucumber, 3 didn't have any copy space and 1 didn't have tomatoes or cucumber, (and wasn't keyworded incorrectly), although it did have space for text.

I guess it will be a while before the kinks are ironed out.

Shelma1 · « **Reply #4 on:** October 12, 2017, 15:23 »

Meh. I tried a search and got all photos, so I added the term vector. Then I got a bunch of vectors that had nothing to do with the other search terms. Easier to use regular search.

eyewave · « **Reply #5 on:** October 13, 2017, 01:58 »

Quote from: Jo Ann Snover on October 12, 2017, 11:13

Just playing around with it a bit suggests that it could grow into something interesting, although it might need some rethinking about what search input is. For example, if I wanted an image with a woman on both the left and right of the image and copy space in the center, I can't use the little box where I move around tokens to find that as I only get one "woman" token for the single search term.

Tried the same search.
You can add the keyword "woman" twice - and walla, there's two women tokens and then two women in at least some of the returned images, two images were exactly like what you asked for.

Jo Ann Snover · « **Reply #6 on:** October 13, 2017, 20:49 »

Some more press:

https://www.engadget.com/2017/10/12/shutter-stock-composition-aware-search/
("For when you absolutely need a photo with a dog in the lower left corner.")

https://siliconangle.com/blog/2017/10/12/shutterstocks-new-image-search-uses-ai-let-search-image-composition/

http://www.itbusiness.ca/news/check-out-shutterstocks-new-ai-powered-image-search-engine/95758

YadaYadaYada · « **Reply #7 on:** October 16, 2017, 14:52 »

Sounds like the same AI and smart search that they announced before. That one went for shapes, colors and little on words. A log on a river would come up the same as a dog on a river, brown on blue, if they were similar compositions. I'm not sold yet, on either of these. Press release garbage.

Oh, looks like the same search, just enhanced and current news. https://venturebeat.com/2016/03/12/shutterstock-shows-machine-learning-smarts-with-reverse-image-search-for-stock-photos/ March 2016 version. Similarly, Shutterstock developed its own convolutional neural network for its reverse image technology, something that’s also being used to improve its “similar image” option, which is available at the bottom of each image result.

Pauws99 · « **Reply #8 on:** October 17, 2017, 10:08 »

Quote from: YadaYadaYada on October 16, 2017, 14:52

Sounds like the same AI and smart search that they announced before. That one went for shapes, colors and little on words. A log on a river would come up the same as a dog on a river, brown on blue, if they were similar compositions. I'm not sold yet, on either of these. Press release garbage.

from one press release "Proper nouns don't work so hot. Searching for "Beyonce" resulted in pictures of (mostly) Caucasian women, and even one of a lady wearing a whipped cream bikini. " yeah I'll settle for that!

MicrostockGroup Sponsors

Author Topic: Composition Aware Search tool announced (Read 5154 times)

Jo Ann Snover

Jo Ann Snover

Pauws99

kuriouskat

Shelma1

eyewave

Jo Ann Snover

YadaYadaYada

Pauws99

Related Topics

Sponsors

Microstock Poll Results

Sponsors