MicrostockGroup

Agency Based Discussion => Dreamstime.com => Topic started by: ozbandit on August 01, 2007, 17:17

Title: Dreamstime down ?
Post by: ozbandit on August 01, 2007, 17:17
Is the site down for everyone else ?
Title: Re: Dreamstime down ?
Post by: hospitalera on August 01, 2007, 17:18
yep, for me also and if you look over there>>>>>>>>>>>>>>>>

SY
Title: Re: Dreamstime down ?
Post by: madelaide on August 01, 2007, 17:22
Yes, it's not working. StockXpert also, but at least I get a page saying it's in maintenance.

Regards,
Adelaide
Title: Re: Dreamstime down ?
Post by: hatman12 on August 01, 2007, 17:22
Haven't you noticed?  It goes down every night about this time.  I know, because it coincides with my morning upload time here in Australia - about 8.00am Australia time, Dreamstime goes off-line for an hour or two.  Every day.
Title: Re: Dreamstime down ?
Post by: Pixart on August 01, 2007, 17:43
Son of a %#!h!   :'(  I've got about 48 hours to finish a website for a client and I can't download the photos.  This happens each and every rare time I have a design job..... UGH.  I'm sorry folks.  It's my fault.
Title: Re: Dreamstime down ?
Post by: stokfoto on August 01, 2007, 19:09
down here too,and it's been more than an hour,so I don't think it's something planned before hand,I mean perhaps not a maintenace work.
Title: Re: Dreamstime down ?
Post by: sharply_done on August 01, 2007, 19:41
I think Freezingpics's new hot seller caused the outage: he was selling it so often that it singlehandedly crashed the server farm ... what nerve!
Title: Re: Dreamstime down ?
Post by: madelaide on August 01, 2007, 21:24
Haven't you noticed?  It goes down every night about this time.  I know, because it coincides with my morning upload time here in Australia - about 8.00am Australia time, Dreamstime goes off-line for an hour or two.  Every day.
I don't observe this problem.  I check sites at night (about 6-7pm my time, GMT-3), and rarely have a problem with DT (actually rarely with any site).

Regards,
Adelaide
Title: Re: Dreamstime down ?
Post by: ozbandit on August 01, 2007, 23:06
First time I've seen DT go down like that in past two months and I also check it pretty regularly. This actually looks like a pretty major outage, that could have been caused either by a DOS or hardware failure. After all it's been down for almost eight hours now and no signs of revival any time soon.
Title: Re: Dreamstime down ?
Post by: chellyar on August 02, 2007, 01:04
Sunspots?  :)

A bit odd that a large 'E' business wouldn't have a contingency with a secondary host to put up a 'Sorry, we're broke' message, a bit like the StockXpert one.

(Also odd that StockXpert should be down this long for a planned upgrade, but that's should be in the other forum.. :) )
Title: Re: Dreamstime down ?
Post by: Achilles on August 02, 2007, 01:04
We had a serious hardware crash and our engineers are currently restoring the system and uploading backup files. Some of the data seems to be permanently affected, but we try to limit the disaster as much as possible. As it looks it will take at least a few more hours, this is definitely the most severe accident that happened.
We are sorry for the downtime and hope to be back online soon. Thanks for your understanding.
Title: Re: Dreamstime down ?
Post by: fotografer on August 02, 2007, 01:07
Thanks for letting us know.  I hope you manage to get it all sorted out soon.
Title: Re: Dreamstime down ?
Post by: hatman12 on August 02, 2007, 01:12
Thanks Achilles.

How professional.

FT could do with taking a few customer service lessons from Dreamstime.......
Title: Re: Dreamstime down ?
Post by: ozbandit on August 02, 2007, 01:17

Honesty is a wonderful thing, thanks for posting an update in here and good luck in getting it sorted. Having managed large networks and servers I know where you guys are coming from so no pressure.
Title: Re: Dreamstime down ?
Post by: rene on August 02, 2007, 03:32
Thanks Achilles.

How professional.

FT could do with taking a few customer service lessons from Dreamstime.......
Very professional... and nice blank home page :-\.
I start to worry about all this microstock world. It seems to be little amateurish. No clustering, no fast backup, no informations.
Title: Re: Dreamstime down ?
Post by: chellyar on August 02, 2007, 06:03
rene has a valid point...

I don't want to knock DT, and I appreciate that Achilles has posted to this forum to tell us what's going on.  But.  (There's always a but).

DT's DNS servers are currently off the net, which means they have either lost quite a bit of hardware, their connection to the net, or there is no redundancy in their connection to the web and a $5 cable has failed.  (Or they didn't pay their hosting bill, but I doubt that!  :) )

The actual fault is academic, if I were part of DT management I'd be asking where the heck is the redundant offsite DNS so you can at least redirect to a 'Please call back later' page?  For a business that is 100% online this does seem like a major failing.

My 'Day job' is as an IS manager, and I used to own and run a web hosting business, so I feel qualified to say that the DT IT staff need a good smack on the back for the head for dropping the ball somewhere.

Yes it can take a long time to fix things and restore 100's of Gb of data, yes you have to wait for vendors to supply replacement gear..   But tertiary DNS and hosting with a reputable hosting company costs a few $ a month, and would have saved a great deal of lost face.

We already saw another microstock company demonstrate an apparent total lack of project management methodology and IT maturity with an upgrade gone wrong this year.

Sitting on the fence, as I am, I suspect this a case of IT centric companies not paying enough attention to how most large businesses treat DR and business continuity in their systems.

Just my 2c worth...
Title: Re: Dreamstime down ?
Post by: Achilles on August 02, 2007, 06:23
Chellyar your thoughts are shared. Dreamstime's  severs are located in what is advertised as the biggest provider in the US, with state of the art technology.

I am afraid that I cannot share how we are structured but all requirements and standards are met on our side, with backups put into place and proper disaster recovery. The problem doesn't reside on our side, nor on our own techs team. What we do is to have everyone involved in fixing this problem ASAP as well as ensuring this will not happen again.
Title: Re: Dreamstime down ?
Post by: Beach Bum on August 02, 2007, 08:04
Some of the data seems to be permanently affected?  Don't like the sound of that!
Title: Re: Dreamstime down ?
Post by: HughStoneIan on August 02, 2007, 08:26
... serious hardware crash..... data...permanently affected.....disaster......the most severe accident.....

Thanks, Achilles, but YIKES!!
Title: Re: Dreamstime down ?
Post by: ptlee on August 02, 2007, 09:07
Yikes! I just requested for a payout yesterday.

I hope everything turns out fine.
Title: Re: Dreamstime down ?
Post by: melking on August 02, 2007, 09:08
oh my!  this does not sound good
Title: Re: Dreamstime down ?
Post by: Beach Bum on August 02, 2007, 09:11
Yikes! I just requested for a payout yesterday.

I hope everything turns out fine.

Yeah.  Same here. 
Title: Re: Dreamstime down ?
Post by: madelaide on August 02, 2007, 15:55
I have also asked for a pay-out a couple of days ago.

Let's hope the extent of the data loss is small.  I suppose a good system won't lose more than a few minutes of new information.  As long as they haven't hired the same IT people as FT did....

Regards,
Adelaide
Title: Re: Dreamstime down ?
Post by: Beach Bum on August 02, 2007, 16:28
I was all set to buy a new camera after this payout.  Oh, well.  Guess it'll have to wait. 
Title: Re: Dreamstime down ?
Post by: nativelight on August 02, 2007, 17:49
Chellyar your thoughts are shared. Dreamstime's  severs are located in what is advertised as the biggest provider in the US, with state of the art technology.

I am afraid that I cannot share how we are structured but all requirements and standards are met on our side, with backups put into place and proper disaster recovery. The problem doesn't reside on our side, nor on our own techs team. What we do is to have everyone involved in fixing this problem ASAP as well as ensuring this will not happen again.

Achilles, check out Rackspace.com.  Our company (my day job) has used them for more than five years without a single incident.  They have fantastic customer support.  Of course if it's Rackspace you're using forget my suggestion.  :o
Title: Re: Dreamstime down ?
Post by: Beach Bum on August 02, 2007, 19:54
Dreamstime is back online! 
Title: Re: Dreamstime down ?
Post by: ptlee on August 02, 2007, 20:09
Phew. At first glance mine looks ok, the earnings history still shows my request as pending, so should be ok.

Perhaps give DT a bit of time to sort things out.

Must have a been a real scare for everyone, including DT staff.

ptlee
Title: Re: Dreamstime down ?
Post by: HughStoneIan on August 02, 2007, 20:19
Perhaps DT could hold a Disaster Recovery workshop and invite FT to be their first student :)
Title: Re: Dreamstime down ?
Post by: sharply_done on August 02, 2007, 22:04
FYI: The FTP site isn't working yet.
Title: Re: Dreamstime down ?
Post by: a.k.a.-tom on August 02, 2007, 22:11
Whoa.... mine looks fine!!   Don't mind saying I was sweating that one, I just hit a payout point this week.    8)-tom
Title: Re: Dreamstime down ?
Post by: pr2is on August 02, 2007, 23:31
Data seems to be OK, but I don't see thumbnails. Rightclick - Show Picture doesn't help. Probably takes more time to restore.... unless everyone else sees them?
Title: Re: Dreamstime down ?
Post by: hatman12 on August 03, 2007, 01:05
Ummm.... not sure this is what you want to hear........ I see thumbnails for all my images.
Title: Re: Dreamstime down ?
Post by: sharply_done on August 03, 2007, 01:10
Uh-oh, thumbnails and images are mostly missing ... for everyone!

Although thumnails show up my home page, most are absent when I do a search.

I guess this means a few days of zero sales on DT.
Title: Re: Dreamstime down ?
Post by: Bateleur on August 03, 2007, 03:00
The thumbnails are there in my 'Online files' page, but when I try to look at any one image it doesn't show. All the data shows fine down the right-hand side, but no image.

Then, if I try to look at images when not logged in, they appear, but overlaying a lot of the text on the page ... creating a mess.

This is probably gonna hit sales   :(
Title: Re: Dreamstime down ?
Post by: fintastique on August 03, 2007, 04:03
i did a few searches and there were thmbnails.

I managed to FTP the backlog of images but they haven't appeared to be keyworded. fingers crossed.

I actually managed a couple of sales since this mishap
Title: Re: Dreamstime down ?
Post by: hospitalera on August 03, 2007, 04:14
Had a sale today, so it seems they are getting back on track *insert relieved face here* SY
Title: Re: Dreamstime down ?
Post by: Achilles on August 03, 2007, 05:53
Hi guys,

Thanks for the patience, everything is back at full speed. Sorry for not posting here earlier, we really needed some sleep.
Priority was given to downloads/uploads area. There are some losses, but we consider them minor. Minor doesn't mean none, another lesson learned.

Some of the thumbnails were not available for a couple of hours only to some users, until the caching propagated everywhere. We could've waited until this was done, but it would've been taken longer. While it affected only a part of our members (including our RO office). US seemed to be fine.

We are still working on lots of things in the backups trying to limit the loss, thanks once again for the patience. 
We will post further announcements in this thread: http://www.dreamstime.com/thread_7982 (http://www.dreamstime.com/thread_7982)
Title: Re: Dreamstime down ?
Post by: HughStoneIan on August 03, 2007, 07:52
Achilles,
Thanks so much for making the effort to give us prompt updates. That is greatly appreciated.
Title: Re: Dreamstime down ?
Post by: snem on August 03, 2007, 08:52
And.... what about payouts??? :-P
Title: Re: Dreamstime down ?
Post by: Achilles on August 03, 2007, 10:41
Payouts are sent as usual. They are sent every few days, we cannot send them daily, they're way too many ;)
Title: Re: Dreamstime down ?
Post by: ptlee on August 03, 2007, 13:43
Hi Achilles,

I can understand how stressful it must have been during the recovery, good job. Get a good rest, and learn from this and come back stronger.

Got got my Paypal notification of my DT payout, thanks!

cheers,
ptlee
Title: Re: Dreamstime down ?
Post by: sharply_done on August 03, 2007, 16:39
A little off topic, but related to the posts about payouts: I request my payouts about a week before the month ends so that I have the money in my bank by the end of the month.
Title: Re: Dreamstime down ?
Post by: hospitalera on August 03, 2007, 16:42
My ftp'ed images appeared in the queue but seems to have doubled, anybody else? SY
Title: Re: Dreamstime down ?
Post by: HughStoneIan on August 03, 2007, 19:27
Double for me too, but I'd rather see double uploads than none at all.
Title: Re: Dreamstime down ?
Post by: takestock on August 04, 2007, 13:06
Site down again.

It's been up and down for a while now.
Title: Re: Dreamstime down ?
Post by: stokfoto on August 04, 2007, 14:28
it's working for me but I suspect there is something wrong with keyword searching functi,on.because one of my submission which is accepted after the down time doesn't seem to be included  by the search engine and it has been several days since it was accepted ,which means it should be found by typing the keywords.anyone experiencing the same?
Title: Re: Dreamstime down ?
Post by: Pixart on August 04, 2007, 15:01
I can't seem to find my own statistics page.  Is it the alzheimers, or is it really gone?
Title: Re: Dreamstime down ?
Post by: madelaide on August 05, 2007, 11:55
I can't tell about yesterday, but statistics are there now, the last button in Management area.

Regards,
Adelaide
Title: Re: Dreamstime down ?
Post by: Tomboy2290 on August 06, 2007, 17:36
The site goes up and down and up then down ... but the sales are still happening somehow. Requested a payment late last week and got it 24 hrs later. Seems they are having a rough time of it but doing the best they can. I hope they get things smoothed out soon.
Title: Re: Dreamstime down ?
Post by: pr2is on August 06, 2007, 18:30
I am pretty sure they will. Remember doom and gloom prediction we heard during Fotolia's upgrade that went awry, and Fotolia survived - and DT is nowhere in such deep doodoo and is better organized in many regards (it's not a stone in FT's garden by any means).

All this feels like growth problems in young industry, it all will sort itsefl out. Remember how many coasters first attempts to burn CD produced? Who gives it a second thouight now, 8 years old breeze through it...