Hi All,

Thanks for your patience - I'm now back in New York and able to respond better.

First, I think it's important to look at numbers.  We added over 3 million "accepted" images in Q1 - so obviously, images are getting accepted at record rates.  As I've stated, I can say that quantitatively, the number of complaints that we get is very small relative to the millions of images being reviewed - of which, accepted images are only a part. 

But let's hypothetically say that it's 10, 100, 500 or more disputed reviews out of millions of images processed.  It doesn't matter.  That could be 10, 100, or 500 people who have done work and gotten a result that they either: a) don't agree with; or b) don't understand.  We consider it an obligation to get it right all of the time, or at least do a better job of explaining the review.  One of our key internal measurements is contributor happiness and satisfaction and people aren't going to be happy if they feel that they didn't get an accurate review.

I've looked at a number of these personally.  I've explained in other posts that disputed reviews fall into three categories. 

1) The review was incorrect (i.e., the contributor is 100% right to complain).
2) The review was borderline (i.e., if you were liberal or conservative in interpreting our standards, you could argue the review either way). 
3) The review was 100% correct, but maybe not fully understood or agreed with.

In scenarios #1 or #2, we generally approve on a 2nd pass. We appreciate that contributors have invested their time, creativity and money in creating content, and the goal is of course to take re-review scenarios as close to zero as possible.

In terms of reviewers having an incentive to reject to earn more money - they don't (nor would we accept that).  Our review queues are always receiving new images and submissions are at an all-time high.  We also track a number of metrics related to reviewer activity which would result in seeing that behavior in the numbers or in feedback. 

Obviously, if and when we find an issue -- either internally on our own or via contributor feedback -- we address it.  We have a number of internal ways that we track review performance and the patterns of our reviewers, as well as extensive training programs.  Those things are built-upon daily and weekly and reviewers regularly interact with our coordinators.

I'm personally grateful for the hard work of our reviewers, most of whom come from photo, illustration, and design backgrounds.  They're quite skilled and many have been with us for years.  They're asked to get it right 100% of the time, which is no easy task across thousands (literally millions) of individual judgments. 

We have a large team of reviewers.  If there's an issue, it could be one or two batches out of hundreds or thousands being reviewed.  The best thing has always been to contact us and we'll address it. 

We greatly respect your hard work and welcome both criticism and feedback.

Thank you!

Best,

Scott