I have a counterexample to offer.  I've been submitting photos from a few recent shoots in small batches of 20 images.  Most have sailed through with 100% acceptance.  One batch was completely rejected; I complained, was advised to resubmit with a note, and had them all accepted the second time. 

Two batches had significant numbers of rejections (around half); when I looked very closely I could see that I had some color fringing.  I had two lights blowing out the background (these were studio shoots of a model), and the combination of the high power setting and the lens I was using created that fringing problem.  Easy to fix in Camera Raw; I'll resubmit and expect them all to be accepted.

My point is that even though I thought my technique was fine and I was having a high rate of acceptance, I could still mess things up on occasion.  The reviewer was right; the problem wasn't dramatic, but it was there.

Quote from: robhainer on May 31, 2014, 08:06

Reviewers shouldn't be active contributors at the same time. Their ports should be disabled if they are reviewing. There's a clear conflict of interest for reviewers who have the power to keep out images that would compete with theirs.

I agree but I don't think that is the case. I think I read somewhere that you can be both.

You are correct..At least thats the way it was when i reviewed for 3+ years. It was a requirement.

There is a very good constructive thread going at SS where scott is posting. If you have issues Pls post them. Speak from your heart Pls. Nothing personal. We are finally having a conversation.

Thanks

http://submit.shutterstock.com/forum/viewtopic.php?t=130946&postdays=0&postorder=asc&start=0

I can't as I am banned. And I don't miss it one bit over there.

Well I suppose we should be at least grateful a response from a SS admin was made.

Albeit, just a collection of words.

462 words to be precise, just words, deflecting completely the concerns of contributors in the current climate of hopelessly wrong reviews and a very strong feeling that something has significantly changed in how our images are reviewed.

462 words that state the complete obvious and restating information what everyone already knew.

There is a plethora of experienced contributors that cannot get a single image approved at the moment and Scott chooses to completely ignore the problem, even more so, fails to even acknowledge that current concern in his response. Amazing.

Yes, we can all write a ticket to support to have our images re-reviewed and yes, we don't mind doing that on the single digit occasions it used to occur when a reviewer made (in our view) a mistake. Occasionally, the reviewer was right and there was something wrong with the file which on closer inspection was corrected. That's fine, that's why there is a review system. It's in everyones ultimate interests that the standard of imagery within the SS library is the best it can be, we all want to attain higher and better standards to ensure customers keep loyal to SS and keep buying quality images. That, in my view, is a given.

But the current status of image reviewing is totally different situation. This is wholesale rejection, in some cases, of entire batches of images from contributors that for years have submitted images to SS and in many cases have the best selling images in the entire collection.

Contributors of such experience do not complain unnecessarily about such matters unless something is seriously wrong, that is also a given.

SS has changed something significantly and for reasons that only they can explain, appear unwilling to divulge to contributors what it is.

Is it the migration of ex- IS contributors into the SS review teams that notoriously 'scupper' other contributors submissions that compete with their own images, or is it a completely new influx of reviewers that are simply incompetent. I don't think so.

I think it might be something quite different.

Conspiracy theorists makes cup of tea and settles down to read on

SS need to cut costs to maintain profits, quarter by quarter, year on year. They have stakeholders now, and we all know, shareholders demand more and more, are never satisfied and always love to see cuts. Lower costs mean higher dividends. Yummy.

"With a large team of individual reviewers, over 55,000 contributors and millions of images passing through this process, every single case is different"


A large team of individual reviewers - that costs. OK they make only a penny or two per image, but when you calculate the number of images reviewed every week that is a cost that's going to add up PDQ. We know that SS add around 245,000 images a week to their collection. Each of those images have been reviewed and pennies paid out. But you have to consider all the images that were not approved that week which were also paid pennies for.

If 245,000 pass how many do you think fail? 500,000, 750,000? They all have the same associated cost. Ouch.

If you have such an overhead which you're paying for every week (pass or fail) it would make sense for an 'automated system' for reviewing to be considered. Not just to reduce your costs but also to improve efficiency and maintain quality standards.

In fact, I think, an automated system to 'weed out' substandard images in a 'primary review' process is in fact the Holy Grail to a company like SS, indeed any large digital image library. Imagine if you had an automated system that kicks out all the cr*p before it gets to a 'human' reviewer. If the 'failed' numbers are in the region of 500,000 a week imagine the cost savings in having an automatic image filtering system in place. The human reviewers only get to see (and be paid for) images that have passed a preliminary review and your costs for reviews just dropped by 65-80% - depending on the numbers that fails anyway.

Now consider this.

Beta Testing.

I know that's the buzz term of the moment with DT  but this might be a different type of Beta Test. A SS Beta Test.

Software to review images is probably still in it's infancy, or the early adopters phase of development. It needs to be tested, no doubt it has been through lab tests now it's a real test (aka a beta test) on real images. Then it can be really tested, reviewed, results analysed, software can be tweaked and then retested again. But you probably would not want to tell people you're doing it. Why would you? You don't want the likes of Getty or FT/DT to know what your doing and you want to test the software under 'normal conditions' of image submissions. So you say nothing.

This software would give you a real competitive advantage in cost saving terms, you're saying nothing.

You just start using it.

Maybe the software is not able to precisely determine the reasons for non-approval (yet) so rather than a single reason being given, multiple reasons are given, WB, Focus, composition etc. Ring any bells?

OK it will cause a small amount of disruption in the short term, experienced submitters will get angry and confused, people will complain, some may even come up with conspiracy theories but "we need to look at the bigger picture guy's think of the money we will save in the long term" SS will say.

And then it all stops and goes back to normal again. Everyones happy and just believes Attila finally gone on holiday and it's over.

The Beta Test program will have a defined start date and finish date




Of course I could be wrong, maybe the reviewers are just ****

IMHO it's now blindingly obvious that this is exactly what's happening, and that SS won't discuss it or even acknowledge it.  At some point, though, I think they'll have to.  For one thing they'll want to trumpet the 'success' - i.e. cost savings of their new automated reviewing tools to investors.  Or some former employee or contractor will leak it.   

"Or some former employee or contractor will leak it.   "

If they do leak it out they will have to move to Russia 

got slammed again for poor lighting. Time to take a break from this ordeal...

I am nearly at the point of giving up completely. This is getting so freakin' old.

Ok, Shutterstock just handed me a very crazy review. A couple of images that I shot from a tripod at ISO 100 and f/11 have been rejected for noise and incorrect focus. I'm a little confused as to what more is required to get an image accepted there.

Is there a way to get them to revaluate these images?

All this being said about Shutterstock reviews, and I've had a bone to pick with them every once in a while. I haven't had issues lately, but I can tell when they hire a bunch of new reviewers. I think it's more or less worked itself out now. They are still more consistent than reviewers on other sites like 123RF, Canstock and Fotolia who might throw out a weird rejection that makes no sense, and you know they don't know what they're doing.

I personally haven't had problems with acceptance. I've been restoring 1800's photos and uploading them as editorial for quite awhile now since I sorta gave up on the traditional stock photos. No problem there. Tuesday I uploaded some of the new grunge style photos that they were wanting. I hadn't tryed to upload them before and was holding my breath about them being accepted, but all of them were except for one. These grunge photos would have never been accepted before if they hadn't been textured with grunge. I have them on my Fine Art America site.

I had a couple random images approved today out of a set that had previously been 100% rejections. While I am not complaining, it is odd that the rest of the groups was rejected then these went right through.

Hello Ethan,

Thanks.  I'm happy to respond, but unfortunately I'm traveling at the moment with intermittent access to wifi and email.  This week I was with members of our team in London to meet with some of Shutterstock's European contributors directly, as well as in Berlin to (also) meet with contributors and attend the annual CEPIC conference. 

My apologies for any delay, but these trips are for the purpose of speaking with our contributors and partners to gather their input and feedback. 

All of what I have posted about Review remains true and accurate.  Shutterstock is a technology company, but as we've already stated publicly in our blog posts, humans review images.  I believe others have already pointed to the reviewer job descriptions on our site, and the use of reviewers is well-established in the industry. 

Given that we've posted previously on these questions and we've asked contributors to contact us directly anytime they have an issue, what information are you most looking for so I can best answer you? 

As stated, we're formulating a longer FAQ.  We're also creating and releasing much more educational material (often in multiple languages) so that our review standards and practices are well understood. We're increasing the types of content that we accept as well. 

The illustration review post was one example, as was our post about editorial illustrations. As another, please see our 80-page "protect your content" guide, which addresses things such as editorial, credential, trademark and copyright standards:
http://www.shutterstock.com/blog/protect-your-content.

Happy to be of help.

Best,

Scott

I appreciate the reply, I have not checked (yet) but I hope a similar response has been posted on your site, at Shutterstock.

Firstly, I do appreciate the restrictions that are imposed during international travel. For nearly ten years straight I was hardly ever at home during the week or even in my own country. However with todays more advanced technologies, WiFi is standard especially in Europe and most certainly in London. I assume your staying in hotels?

I note your response to my points and no doubt to my earlier posted 'scenario'. You must have read that (in your homework prior to responding) to make your comment about humans.

I was speculating that the first pass review might be automated, which you delicately avoided commenting on. And I can understand why.

I accept that, as you say yourself, Shutterstock is a technology based company. I get that. I still believe the Holy Grail for a company like yours is an automated systems at first-pass to exclude images from 'human review' in order to reduce your costs.

It is a fact you have to pay a reviewer (albeit pennies) to either pass or fail an image when they review. The image that passes will 'pay back that cost' as soon as it begins to sell.

However, a failed image is dead money.

More images fail than pass, therefore the dead money cost has to be higher than 'passed money'. That's simple math. To get those images reviewed in an automated fashion and off the system makes sense.

You still continue to state that contributors can write to SS and appeal against image rejections. We know that. We do that already.

I should not have to labor this point. There has been a sea-change in recent weeks. The effect has been huge. It has effected many, most notably some of the most seasoned, experienced and successful contributors you have. Rejections (for multiple reasons) that make no sense. When appealed those same images are accepted.

We are as a group of experienced contributors not stupid. Give us a little credit. Something has changed in the review process (at this moment in time) and we know it has as a result of the wholesale rejections that have occurred the very recent past few weeks.

I know from my own experiences in the technology business that often 'being first' is key. I also appreciate that if your working on a new technology, no doubt through a third party, confidentially is essential as well as a necessity.

The less you say, the more I read your difficulties, I do understand that.

I just hope whatever review is going on it can be perfected quickly so everything can get back to normal asap.

And thanks again for getting back and enjoy my home town of London

PS The weather tomorrow is supposed to be rubbish.

Quote from: ethan on June 06, 2014, 11:48

So it's Friday, end of the week, time to tie up all those loose ends. Well not the case for Scott Braut it seems.....

Following an avalanche of posted concerns on the SS forum relating to the unfounded, ridiculous and down right stupid rejections of perfectly good images (from established contributors mind, not the cr*p from newbies) Scott finally succumbed with a stalling post (attached)..

That was Monday, five days ago.

He obviously does not have the capacity (or is it the will and desire ?) to respond to the glaring question of what is going wrong.

Perhaps he does not have mobile communications, strange that as he essentially works for a digital based company.

He was 'forced' to respond last time following a (my) post on a public forum last week, (which they hate btw) so here we go again.

Scott Braut - Do your job. It's why you take your salary each month.

Respond.

The guy already responded. I'm not sure what you'd want him to add. He said that if you have an issue with a rejected image to contact support, and they will look at the image again. They will either overrule the previous rejection or uphold the rejection. I know the system works because I've used it twice recently and had rejections overturned.

He also told us about several areas where they are trying to get better so we don't have to go through the initial anguish of a bad rejection.

Thanks for stating the obvious. The appeal process your so keen to mention only proves my point that something is amiss.