I haqve wondered that as well . I suppose if it would help to ease the load on the entire system,IS's it folks would shut the forums down. But, not being an IT expert myself, this is purely speculation on my part. Hope it is all resolved soon!

Google, Yahoo, and Wikipedia all use MySQL and all of them are WAY larger and don't have these issues.  Why is that?

Maybe it has something to do with accounting/bookkeeping/transactions - the sites you mentioned aren't retail outlets.

I wonder how much traffic is due to contributors just playing around (forums,...)

Shutterstock seems to do things differently, we go to a different site to check stats and play on the forum.  In my brief experience at SS, there hasn't been any of these monstrous troubles the others eventually stumble into that I've noticed.  I wouldn't be surprised if contributors spend more time than purchasers on these sites.

Quote from: sharply_done on September 25, 2007, 12:51

Quote from: StockManiac on September 25, 2007, 12:42

Google, Yahoo, and Wikipedia all use MySQL and all of them are WAY larger and don't have these issues.  Why is that?

Maybe it has something to do with accounting/bookkeeping/transactions - the sites you mentioned aren't retail outlets.

IS is trying to blame MySQL for not being able to handle the amount of transactions that they have.  Here is the exact quote from IS: "We ran up against a known bug in mySQL 5 (which is what the site runs on). We simply can't process any more transactions per second."

To a database, a transaction is a transaction is a transaction.  A database doesn't know (or care) whether you are buying, selling, storing info, seeking info, etc.  To a database, it really doesn't matter whether you are storing text, numbers, currency, images, sound, signatures, etc. It's still all a transaction.

It's the application that discerns the difference in what is being stored in the database (via the business rules layer of an n-tier system).

Google and Yahoo definitely have WAY more transactions than IS, yet their implementation of MySQL doesn't seem to have these issues.

Thus, the obvious conclusion is that it is not the database, but it is their application code that is the problem.

BTW, both Google and Yahoo do have sales transactions.  Check out Google Products (http://www.google.com/products) or Yahoo Shopping (http://shopping.yahoo.com/).

i don't know much about programing things but i heard someone say before (perhaps it was on here somewhere) that this is the same problem friendster  ran into a little while ago... maybe someone else has more info 

i don't know much about programing things but i heard someone say before (perhaps it was on here somewhere) that this is the same problem friendster  ran into a little while ago... maybe someone else has more info 

Yeah, that was me. Friendster ran into this exact problem. If someone wants a technical explanation just p.m. me.

Quote from: leaf on September 25, 2007, 16:58

i don't know much about programing things but i heard someone say before (perhaps it was on here somewhere) that this is the same problem friendster  ran into a little while ago... maybe someone else has more info 

Yeah, that was me. Friendster ran into this exact problem. If someone wants a technical explanation just p.m. me.

ah yes.. now i remember. thanks for the post in this thread as well.

Trust me when I tell you that the problem would be with MySQL, which is why I'm pissed! This is a problem they should have seen coming and should have migrated to Oracle. MySQL is not meant for this type of application at this size.

As an iStock exclusive I'm pissed that iStock didn't see this coming because it obviously means they didn't hire anyone with a graduate degree in computer science which should be mandatory for a large company that has a mission critical database.

well, i think all of that is pretty obvious from the ongoing problems they've had since i've been with them, including the implementation (if you can call it that) of the whole disambiguation thing.

all of that points to exactly why i will never be exclusive with them

You might want to stick with what you know, instead of trying to make believe that you understand the technical intricacies of a large database application.
If you read iStock's confession today (which is available @ http://www.istockphoto.com/forum_messages.php?threadid=57849), then you will see that they have run into a problem with MySQL because of a poorly designed architecture.  From IS's own confession: "Can we continue to grow indefinitely with MySQL? Absolutely. We have looked at how other large sites using MySQL have re-architected to handle much larger loads than iStock currently gets. We can see how this would fit into our system - but we cannot re-architect / rebuild the system over night."

Wow, do you have a degree in computer science? Have you worked on large database apps? I did my masters dissertation on distributed graphics rendering (and I used MySQL) before giving up on computer science and going to law school. My question for you is do you actually know about large scale databases or are you just arguing for the fun of it?

About your quote, did you read all the paragraphs above that? The reason for this mornings crash? The 45 minute recover time, that's because MySQL still doesn't have hotbackups or automatic point in time recovery like Oracle or MS SQL. The locking problems wouldn't have occurred on Oracle or MS SQL since they both have row-level locking while MySQL has limited row-level locking, but mainly table-level.

Here is the quote that counter acts what you quoted: "Then there are solutions that promise huge returns, but would require re-architecting the entire system; rewriting a very large portion of the code and database queries while deploying a new server infrastructure. There were reasons why these approaches were not taken seven years ago when the company was starting - with 5 people running a hobby site on a second hand server."

Then the most important one"I am not going to tell you here which approach we are taking - except to say that we are looking forward to growing the site ten times, fifty times, five hundred times the size of what we are today." They wouldn't have said this if they were going to stick with MySQL. Yes iStock would have to rewrite a lot of code because Oracle databases have slightly different calls and many features would have to be completely rewritten to take advantage of the unique features of the large database systems.

If you feel like continuing to argue then I will write a detail technical explanation for you. In the mean time I'll be considering canceling my exclusive contract.

Edit: Oh, and lastly. "To a database, a transaction is a transaction is a transaction" is a complete misstatement since a view is different from an insert, which is different than a deletion. Not to mention MySQL is a database system without proper transactions.