I remove them, so that I do not compete with myself on other sites.
People think that it costs them nothing to have their ports floating here and there.

It might cost a lot. In sales you never get paid for, in undercutting other agencies, and in strange and random even illegal distribution of your images.

 Also I like to know that im not earning anything at that place, and therefore not have to come back and look if there is a payout.

I usually leave my images but if the site can't make at least an annual payout, I will probably remove them.  I usually leave my images because it's taken time to get them on the site and there might be the possibility that things will improve in the future.

I usually leave my images but if the site can't make at least an annual payout, I will probably remove them.  I usually leave my images because it's taken time to get them on the site and there might be the possibility that things will improve in the future.

+1 
After all the work of uploading I don't want to undo it when there is a possibility that the site may turn around.  Leaving images is a way to monitor success and if they start selling again I will resume uploading.  Veer is one of those on the edge for me - after slow and weird review results plus slow sales I get annoyed and take them off the list, then will get some string, high-value sales and put them back on.  Right now they're on, but my last batch was 100% rejected and I've had no sales so far this month so they may be off again in January.  Crestock is one where I have stopped and started but have now stopped for a long time - with a 20% commission and 25-cent subs they will have to generate phenomenal sales to get me interested again.  Featurepics is another one that I have stuck with - sales are slow, but prices are reasonable and their commission is 50%, uploading is easy and reviews are fast and consistent.  They will stay on the list as long as they generate some sales.  I stopped uploading to Cutcaster a couple of years ago due to no sales ever but have kept images there to monitor in case they improve.  Nowadays companies can go in and out of favor quickly and I like having something ready to go just in case.

Remove all images. 

I shop at certain markets because there are certain types of groceries that I like to buy...if that particular product isn't in a market, then I take my business elsewhere.  Price does not usually matter when I'm looking for a certain product with a certain quality (i.e. I prefer to get my vegetables as fresh as possible and I'm willing to drive a few more miles and spend a few more dollars to get the produce I want).

Same applies to the agencies - if the customers see the work they need/want at a different place, then they will shop at a different place.

It depends.  If it's a matter of poor sales (e.g. Zoonar, StockFresh) or ridiculous reviews (e.g. Crestock), I'll leave them up in case something changes or I can make a payout every year or two.  I figure I've already put in the work, and whatever images they have are aging and not of my current quality.  I don't worry about their presence competing with my sales on more successful sites.

On the other hand, sites that screw me (iStock and Fotolia, I'm looking at you) get a slow deletion.  I've removed anything that hasn't sold, or at least hasn't sold recently.  Again, I put in the effort to upload them, what they have isn't my most recent work, and why not squeeze out some last sales while I can?  If reducing my portfolio on iStock from over 3000 images to 576 as of today doesn't let them know how I feel, removing them completely won't get the point across any better.

i've never understood the "dont want to compete with my other ports" argument -- someone who's buying on A is not going to know anything about your ports on B C & D.  and most are not looking exactly  what you have in image x when there are similars elsewhere.  so removing your images from a site has basically 0 effect

the other argument is that leaving images on a site generates sales with no payout, but any site i leave is generating so little income that's of little consequence also

so i rarely actually leave a site, i just ignore them

It's probably a coincidence, but almost every time I delete a site, some other site will step up to fill the void the next month. But, you are right. Some buyers just buy at one site and don't look around.

That said, there are also a lot of buyers out there that come in through Google. So if your image only exists at a couple places, those are the places that search engine traffic will bring up. I think people often underestimate the power of search engines to bring in traffic. Obviously, the agencies don't because they seem to spend a lot on Adwords.