MicrostockGroup Sponsors


Author Topic: Is Shutterstock using software to "first pass" review all images?  (Read 12388 times)

0 Members and 1 Guest are viewing this topic.

« on: July 23, 2015, 16:07 »
+2
A lot, lot, lot of people have been having difficulties with the review process over at SS lately, well really for the last year and a half. I think the reason we are having these difficulties is that SS is using some sort of software to review images.

I'll tell you why I believe this to be true.

One example is if you produce landscape images of ocean waves utilizing neutral-density filters to slow the shutter speed, giving you an image with nice ghosted-waves and a soft dream-like ocean-scape . . .

The software will look at the meta-data embedded in the images, and see the long exposure times and those images will be rejected for Noise just based on the exposure times contained in the meta-data. If the software looks at the image (somehow) and doesn't detect any stray-pixel noise, then the software will look at the ISO data-field and see it is low compared to the exposure time of the image and assign an "Over use of noise reduction software" rejection to the image.

I know that is happening on a number of my images and I suspect it is also happening to a few other contributors as well.

Another example, if you shoot images that are not people and you utilize a very shallow depth-of-field, i.e. a large aperture like f2.8, f1.4, f1.2 etc. the software will somehow assign a "blurry" out of focus rejection just based on the information contained in the meta-data of the image. I think somehow the software "examines" the images and determines the percentage of "sharp" pixels and if the number is below a given number the dreaded "focus" rejection is assigned to the image.

I'm guessing the software is created with a number of "if than" statements, i.e. if this value (exposure time) is above a given number the image is rejected, or the software moves on the the next "if than" statement, if this value (ISO) is below a given number the image has excessive noise reduction software applied and is rejected for that reason, etc.

I believe there are some serious flaws in the software they are using (and I think some of the other agencies are utilizing this software review method as well).

The process of submitting images and having them rejected and resubmitting is time-consuming and wasteful. But, in the minds of SS the process is not time-consuming or wasteful to them because the only person's time that is wasted is the contributors. SS first review doesn't require any time or effort on their part, just a bit of software processing time.

It cost them (SS) nothing in time or energy to have everything passed through this software for it's first review. If your images are lucky enough to have all the right values in all the right fields then they can be moved along to be reviewed by an actual person who will reject your images for an entirely different set of reasons that don't have anything to do with the price of beans in China, but that is a different issue.

So, complaining to someone at SS won't help, if the software gets it wrong 50% of the time, they are still ahead of the game. There really is nothing you could do to change this process.

The best way to combat this "software review" issue is to produce generic looking "stock" images, perfectly lit single items isolated on white backgrounds with 5 thousand dollar full-frame camera bodies with 2 thousand dollar prime lens all to make 25 cents a download . . .

Yeah, that will work, lol.

The problem with this is it leaves no room for creativity, or uniqueness in our images. It ensures that all micro-stock images will look just exactly like all other micro-stock type images, and nobody really wants that.


« Reply #1 on: July 23, 2015, 16:14 »
+20
We need another thread on SS rejections?

« Reply #2 on: July 23, 2015, 16:24 »
+7
I doubt it rejections would be much more consistent if this were the case. Its much simpler they don't have a category for "we don't really like your photo and don't think it will sell"

Rinderart

« Reply #3 on: July 23, 2015, 19:05 »
+4
We need another thread on SS rejections?

I agree. and........."THEY DONT CARE!!!!!!!!!!!! All they know and all that matters to them is 400,000 new ones are  accepted weekly or whatever.

« Reply #4 on: July 23, 2015, 20:27 »
+9
We need another thread on SS rejections?

I agree. and........."THEY DONT CARE!!!!!!!!!!!! All they know and all that matters to them is 400,000 new ones are  accepted weekly or whatever.

Yes, time to move on. Either quit uploading or live with it. Griping about it will achieve about as much as a dog barking at a passing train (Chinese proverb, I think).

« Reply #5 on: July 23, 2015, 21:04 »
0
sjlocke , rinderart, shiyali...
so agree with all 3.
chinese proverb or not , to translate into a language you can understand better...
 THEY DON'T GIVE A RAT'S ARSE
either you protest with your withdrawing any further new uploads
or just move on to another agency.
no point in trying to be like those chinese guys standing in front of the tanks on tianamen square.
shutterstock is not china... they will just keep rolling on as long as you keep rolling over and die
just talking about it.
if you own shitterstockshutterstock stocks, go to their agm and tell Oringer and the other shareholders.  but don't hold your breath 8)

« Reply #6 on: July 23, 2015, 21:11 »
+8
My point was that they are using software to review images, nothing more, nothing less.

Take it however you like, but from what I can tell most people just think it is all done by a bunch of reviewers with "issues" or humans who are just very bad at their jobs.

My point is that in many cases there are no humans involved and the artistic "value" of our images, that we work so hard to create unique pieces of art are reviewed by a software algorithm rather than an actual human.

The fact that a software algorithm is making the determination between "acceptable" stock and "unacceptable" images fits with what everybody is complaining about. I believe it explains why stuff we would consider "poor" quality stock images are accepted and our unique and creative images are rejected.

It's a numbers game, if the "values" contained in the meta-data of the image are in the good range, the image is accepted, if however the numbers contained in the meta-data of the image are out of the "acceptable" range then the images is rejected.

All this happens without an actual human looking at the images we spend so much time and energy to create.

That is my point.

Yes, I know they don't care, I agree that they don't care, but that isn't the point I was trying to make.






Rinderart

« Reply #7 on: July 23, 2015, 23:46 »
+2
No, The point is. your point has been discussed to death about them using a Bot for over 16 Months. thats the point. They will never admit it. and if they do use a Digital review system? Shame on them.There site has become soul less anyway. Maybe this issue is new to you. I understand your concern.


"My point was that they are using software to review images, nothing more, nothing less." Can you Prove this? I cannot and If I could , who cares  thats the biggest point.

« Reply #8 on: July 24, 2015, 02:55 »
+7
His theory is a bit more fleshed out than the average discussion, which amount to shutterstock is using software while offering no details of what image attributes they may be using to screen our images.

His post is well thought out and offers some interesting ideas.

« Reply #9 on: July 24, 2015, 03:10 »
+5
My point was that they are using software to review images, nothing more, nothing less.

Your whole thesis is based on the assumption that the submitted images have the EXIF data intact. EXIF can be stripped through post-processing software, etc. Why don't you submit a few batches without the EXIF data and see what happens, then your assumed "software" won't see exposure time, ISO, aperture, etc.

ShadySue

« Reply #10 on: July 24, 2015, 03:25 »
+3
No, The point is. your point has been discussed to death about them using a Bot for over 16 Months. thats the point. They will never admit it. and if they do use a Digital review system? Shame on them.There site has become soul less anyway. Maybe this issue is new to you. I understand your concern.


"My point was that they are using software to review images, nothing more, nothing less." Can you Prove this? I cannot and If I could , who cares  thats the biggest point.

Who cares?
Presumably all the people who have been "discussing the issue to death over the past sixteen months".  ;)

« Reply #11 on: July 24, 2015, 03:57 »
0
We need another thread on SS rejections?

I agree. and........."THEY DONT CARE!!!!!!!!!!!! All they know and all that matters to them is 400,000 new ones are  accepted weekly or whatever.

And 300,000 of those are ugly illustrations that look all the same.

« Reply #12 on: July 24, 2015, 10:24 »
+4
A lot, lot, lot of people have been having difficulties with the review process over at SS lately, well really for the last year and a half. I think the reason we are having these difficulties is that SS is using some sort of software to review images.

I was making this case months ago and most of the responses dismissed the idea as a conspiracy theory.  A few said "hmmm...".    SS never directly answered the question - they danced around it with weasel words.   In statements to investors around the time they went public, they said they're working on automated reviewing.

I have absolutely no doubt that software is doing a first-pass screen and is responsible for a lot of weird rejections.  Anything unconventional lands on the floor.  It's one reason I quit doing microstock. 

Rinderart

« Reply #13 on: July 24, 2015, 10:31 »
+1
No, The point is. your point has been discussed to death about them using a Bot for over 16 Months. thats the point. They will never admit it. and if they do use a Digital review system? Shame on them.There site has become soul less anyway. Maybe this issue is new to you. I understand your concern.


"My point was that they are using software to review images, nothing more, nothing less." Can you Prove this? I cannot and If I could , who cares  thats the biggest point.

Who cares?
Presumably all the people who have been "discussing the issue to death over the past sixteen months".  ;)

I try to only spend time on things I can change. I care a Lot. But It does no good at all and just makes me feel bad that a lot of us sat by and did nothing years ago.

« Reply #14 on: July 24, 2015, 11:44 »
+2
No, The point is. your point has been discussed to death about them using a Bot for over 16 Months. thats the point. They will never admit it. and if they do use a Digital review system? Shame on them.There site has become soul less anyway. Maybe this issue is new to you. I understand your concern.


"My point was that they are using software to review images, nothing more, nothing less." Can you Prove this? I cannot and If I could , who cares  thats the biggest point.

Who cares?
Presumably all the people who have been "discussing the issue to death over the past sixteen months".  ;)

I think for anyone that still submits to microstock, this issue is very much alive and in play. 

Obviously SS will continue to work on this and "enhance" it because it's seen as an enormous cost saving vs. wages of real reviewers.  But SS seems to have locked themselves into a position of denial. At some point though, the cat will escape the bag and initiate a wide open controversy.  They clearly don't want that, but can't escape it forever.  Wouldn't it be better if they just came clean now and started using feedback from contributors to improve this process?

Rinderart

« Reply #15 on: July 24, 2015, 12:16 »
0
No, The point is. your point has been discussed to death about them using a Bot for over 16 Months. thats the point. They will never admit it. and if they do use a Digital review system? Shame on them.There site has become soul less anyway. Maybe this issue is new to you. I understand your concern.


"My point was that they are using software to review images, nothing more, nothing less." Can you Prove this? I cannot and If I could , who cares  thats the biggest point.

Who cares?
Presumably all the people who have been "discussing the issue to death over the past sixteen months".  ;)

I think for anyone that still submits to microstock, this issue is very much alive and in play. 

Obviously SS will continue to work on this and "enhance" it because it's seen as an enormous cost saving vs. wages of real reviewers.  But SS seems to have locked themselves into a position of denial. At some point though, the cat will escape the bag and initiate a wide open controversy.  They clearly don't want that, but can't escape it forever.  Wouldn't it be better if they just came clean now and started using feedback from contributors to improve this process?

very well said +1 million if I could. or...we find a trick to beat it...LOL

« Reply #16 on: July 24, 2015, 23:48 »
0
Why don't you submit a few batches without the EXIF data and see what happens, then your assumed "software" won't see exposure time, ISO, aperture, etc.

Great idea. I wouldn't be surprised if they reject images lacking EXIF data though, certainly for editorial stuff. Whenever I've got the date wrong in an editorial caption, they're on it like a hawk.


Rinderart

« Reply #17 on: July 25, 2015, 00:14 »
0
Hmmmmmmmmmmmm.

« Reply #18 on: July 25, 2015, 05:52 »
+1
My point was that they are using software to review images, nothing more, nothing less.

Take it however you like, but from what I can tell most people just think it is all done by a bunch of reviewers with "issues" or humans who are just very bad at their jobs.

My point is that in many cases there are no humans involved and the artistic "value" of our images, that we work so hard to create unique pieces of art are reviewed by a software algorithm rather than an actual human.

The fact that a software algorithm is making the determination between "acceptable" stock and "unacceptable" images fits with what everybody is complaining about. I believe it explains why stuff we would consider "poor" quality stock images are accepted and our unique and creative images are rejected.

It's a numbers game, if the "values" contained in the meta-data of the image are in the good range, the image is accepted, if however the numbers contained in the meta-data of the image are out of the "acceptable" range then the images is rejected.

All this happens without an actual human looking at the images we spend so much time and energy to create.

That is my point.

Yes, I know they don't care, I agree that they don't care, but that isn't the point I was trying to make.

your point is a very good one (and bringing this up is too), because apparently no one has thought about it that way yet -- and them using software makes total sense (and matches their overall behavior as well)!

On top, how "smart" or "appropriate for the job" that is is highlighted by the fact that computer vision is a hairy problem to this day, and we (and SS) need to accept that fact. SS' nice little "workaround" is outright inadequate and nothing but backfires.

Yeah, and it's right that they don't care anyway...

« Reply #19 on: July 25, 2015, 05:54 »
0
Why don't you submit a few batches without the EXIF data and see what happens, then your assumed "software" won't see exposure time, ISO, aperture, etc.

Great idea. I wouldn't be surprised if they reject images lacking EXIF data though, certainly for editorial stuff. Whenever I've got the date wrong in an editorial caption, they're on it like a hawk.

that's exactly what will happen -- anyone ever tried logging into Facebook or Google after erasing cookies?! Just have a look at what they're displaying when someone dares using their right to "Privacy" or what's left of it. SS will do the same when EXIF data is missing, as will most-any of those presumably "new economy" outfits out there. You could fake the EXIF entires though (if you haven't got better things to do and from now on out also want to lose time having to "edit" EXIF info too)...
« Last Edit: July 25, 2015, 06:00 by marquixHD »

« Reply #20 on: July 25, 2015, 06:58 »
0
I doubt it rejections would be much more consistent if this were the case. Its much simpler they don't have a category for "we don't really like your photo and don't think it will sell"


Exactly

« Reply #21 on: July 25, 2015, 07:46 »
0

I doubt it rejections would be much more consistent if this were the case. Its much simpler they don't have a category for "we don't really like your photo and don't think it will sell"

or better, from the reveiwers who are in conflict of interest category mind state, they should have a button that says " i don't want your photo to compete with my portfolio".  evident with those experienced contributors who suddenly been told
they do not know how to WB or get the "focus where we think it should be"  ;)

« Reply #22 on: July 25, 2015, 07:48 »
+3
Why don't you submit a few batches without the EXIF data and see what happens, then your assumed "software" won't see exposure time, ISO, aperture, etc.

Great idea. I wouldn't be surprised if they reject images lacking EXIF data though, certainly for editorial stuff. Whenever I've got the date wrong in an editorial caption, they're on it like a hawk.

that's exactly what will happen -- anyone ever tried logging into Facebook or Google after erasing cookies?! Just have a look at what they're displaying when someone dares using their right to "Privacy" or what's left of it. SS will do the same when EXIF data is missing, as will most-any of those presumably "new economy" outfits out there. You could fake the EXIF entires though (if you haven't got better things to do and from now on out also want to lose time having to "edit" EXIF info too)...

That's absurd to the point of making me giggle. Are you aware that there are many ways to create images OTHER than right out of your camera? 3D renders, illustrations, photo composites etc. etc. etc.

There's another thread on here right now about scanning transparencies for stock. When you scan an old transparency, where do you think the EXIF data comes from? If you import 2 or more photos into photoshop and combine them new document as a composite, where do you think the EXIF data comes from? Will these images be rejected automatically? Not really No.

Shutterstock gives a rat's a$$ about EXIF. They don't care what ISO you used, the type of lens, exposure time, shutter speed, time of day or anything else. They care about image quality: low noise, focus and exposure.

Do they have an automated system for pre-screening images? probably. Even your camera already does a lot of the same things the reviewers would look for and doesn't need EXIF to do it. Your camera can tell if the image if in focus. It can tell if the image is exposed properly according to whatever metering mode you're using. Some cameras can even help with composition by identifying human faces. So if a camera can do it (photoshop does all this as well) then a bot can too.

If there are 250,000 images being accepted every week and given the acceptance ratio of the average contributor that means there are upwards of 1 million images being submitted. Unless Shutterstock has an army of reviewers numbering in the tens of thousands, suggesting they pre-screen for the most common errors using software is one of those things that makes me say "well duh".


w7lwi

  • Those that don't stand up to evil enable evil.
« Reply #23 on: July 25, 2015, 13:27 »
+1
According to a post on SS this morning, in the last video session from SS the question of automated reviews came up.  According to SS, they use an automated review when an image first arrives to confirm it meets the 4mp minimum requirement.  After that, all images are reviewed by human eyes.  They also state that they have in excess of 200 reviewers worldwide and they are constantly being reviewed by a team of coordinators.

http://forums.submit.shutterstock.com/topic/76582-review-rejections-information-and-confirmations-from-shutterstock/


« Reply #24 on: July 25, 2015, 17:30 »
0
Why don't you submit a few batches without the EXIF data and see what happens, then your assumed "software" won't see exposure time, ISO, aperture, etc.

Great idea. I wouldn't be surprised if they reject images lacking EXIF data though, certainly for editorial stuff. Whenever I've got the date wrong in an editorial caption, they're on it like a hawk.

that's exactly what will happen -- anyone ever tried logging into Facebook or Google after erasing cookies?! Just have a look at what they're displaying when someone dares using their right to "Privacy" or what's left of it. SS will do the same when EXIF data is missing, as will most-any of those presumably "new economy" outfits out there. You could fake the EXIF entires though (if you haven't got better things to do and from now on out also want to lose time having to "edit" EXIF info too)...

Not correct, while editorial need to have the EXIF date, etc match the description, RF images do not needed the EXIF intact. I have submitted hundreds of images without it. It makes no difference whatsoever as far as reviews go.


 

Related Topics

  Subject / Started by Replies Last post
4 Replies
3799 Views
Last post September 14, 2009, 07:32
by Noodles
29 Replies
5309 Views
Last post March 31, 2012, 17:26
by Paulo M. F. Pires
68 Replies
12683 Views
Last post July 17, 2018, 18:38
by StanRohrer
25 Replies
10313 Views
Last post May 26, 2015, 05:40
by cathyslife
14 Replies
3447 Views
Last post July 23, 2016, 09:28
by etudiante_rapide

Sponsors

Microstock Poll Results

Sponsors