Jan 17, 2006, at 1:32 AM, Jim Popovitch wrote:

I want to say, from an outsider's perspective, that I whole
heartily applaud GoDaddy on the actions they took []

There seems to be a wide split on this topic. I was wondering if
people would privately tell me yes or no on a few questions so I can
understand the issue better.

1) Do you think it is acceptable to cause any collateral damage to
innocent bystanders if it will stop network abuse?

2) If yes, do you still think it is acceptable to take down 100s of
innocent bystanders because one customer of a provider is misbehaving?

3) If yes, do you still think it is acceptable if the "misbehaving"
customer is not intentionally misbehaving - i.e. they've been hacked?

3) If yes, do you still think it is acceptable if the collateral
damage (taking out 100s of innocent businesses) doesn't actually stop
the spam run / DoS attack / etc.?

These are important question to me, and I'm surprised at the number
of people who seem to feel so very differently than I thought they
would feel - than I personally feel. Would people mind sending me
private e-mails with yes/no answers? Longer answers are welcome, but
yes/no will do.

Using the case under discussion as an example, I am wondering why
anyone thinks taking down 100s of innocent domains is a good way to
stop a single hacked machine from doing whatever it is doing? If you
somehow think all that is worth it, take a close look at your cost /
benefit analysis. At this rate, every business on the Internet will
be out of business before we take out even a single moderately large
botnet.

I am also wondering why anyone thinks the miscreant will stop just
because the legitimate owner's domain no longer resolves? Not only
is the machine likely to continue sending spam as if nothing
happened, we aren't even "catching" the guy. I guess you could say
"well, it put pressure on his hosting provider to clean the infected
machine", which is true. I just think that's a bit silly. But maybe
I'm the one who's silly.

Lastly, I wonder what "average" people - people who run businesses on
hosting providers who really don't understand all this computer stuff
- think about such actions. How many 100s of people have we just
alienated for life to stop - er, NT stop - a single zombie? And how
many of their friends are going to hear over an over how the Internet
is not a real business and no one should put any faith in it?

Is this really a good thing?

Tue, Jan 17, 2006 at 02:09:21AM -0500, Patrick W. Gilmore wrote:

Jan 17, 2006, at 1:32 AM, Jim Popovitch wrote:

>I want to say, from an outsider's perspective, that I whole
>heartily applaud GoDaddy on the actions they took []

There seems to be a wide split on this topic. I was wondering if
people would privately tell me yes or no on a few questions so I can
understand the issue better.

1) Do you think it is acceptable to cause any collateral damage to
innocent bystanders if it will stop network abuse?

2) If yes, do you still think it is acceptable to take down 100s of
innocent bystanders because one customer of a provider is misbehaving?

3) If yes, do you still think it is acceptable if the "misbehaving"
customer is not intentionally misbehaving - i.e. they've been hacked?

3) If yes, do you still think it is acceptable if the collateral
damage (taking out 100s of innocent businesses) doesn't actually stop
the spam run / DoS attack / etc.?

I don't think anyone (well ok, anyone sane, I know we have a few nutjobs
on this list :P) thinks that arbitrarily blocking service to hundreds or
thousands of users because someone is unknowingly hacked is an appropriate
way to address network abuse. I really have no idea how aggressive GoDaddy
is with enforcing their AUP, as I don't personally use their services, but
based on what I know about the affected customer and what I can read from
the affected whiner's website I'm certainly not going to jump to the
conclusion that GoDaddy is running around like a hopped up abuse desk
worker on a power trip, shutting off service to random innocent people
because they feel like it.

The question at hand is, at what point does a registrar providing services
have an ethical or moral obligation to step in and do something when they
do encounter an excessive level of abuse by someone using their services?
At what point does ARIN revoke the allocation of a blatant and persistant
spammer who is violating the law without being stopped? I think the answer
is that clearly this isn't something they want to be doing on a regular
basis, any more than an ISP wants to be responsible for filtering every
packet that goes through their routers looking for warez and kiddie porn,
yet I have seen them do it in certain rare and severe cases of unrelenting
abuse.

Maybe it is a judgement call, maybe it isn't. Bottom line, dealing with
abuse is an ass job, and I certainly wouldn't want it. Some days you're
doing a good thing because you shut down a spammer, some days you're doing
a bad thing because you shut down innocent services along with it (and
some days you're just fending off "stop hax0ring me on port 80 or I'll sue
you and call the CIA" e-mails).

I highly suspect that GoDaddy doesn't involve itself in these kinds of
issues lightly, which means that in all likelihood the level of abuse was
severe, with no communication from the person they suspended service to. I
for one have never heard of anyone I know having their GoDaddy service
suspended for this kind of thing. Unless someone has some actual facts
that GoDaddy is engaging in this kind of activity, I'm inclined to give
them the benefit of the doubt. This means, at least for now lumping them
in the "respecting them for taking a stand regarding the abuse of their
service" category, rather than the "wackjob conspiracy theorist
power-crazed zealot" category we all know and love. :)

Patrick W. Gilmore wrote:

Jan 17, 2006, at 1:32 AM, Jim Popovitch wrote:

>I want to say, from an outsider's perspective, that I whole heartily
>applaud GoDaddy on the actions they took []

There seems to be a wide split on this topic. I was wondering if
people would privately tell me yes or no on a few questions so I can
understand the issue better.

1) Do you think it is acceptable to cause any collateral damage to
innocent bystanders if it will stop network abuse?

In some cases. policy is to minimize such. Example: Customer has
a NATted network with multiple machines sharing one global address.
of the machines at customer's premise is causing abuse (virus, etc.)
Null-routing one specific IP address will cause collateral damage to the
non-infected machines at that customer, but I think most of here would
agree that such is justified. , if the impact of the abuse is
minimal, having the customer fix the problem before shutting anything
down is preferred. Another example would be a customer's webserver
which has many name-based virtual hosts, one of which is abusive, and
you are providing IP connectivity. By null-routing one IP you are
causing collateral damage to the non-abusive virtual host customers of
your customer, but I think most would think that justified.

2) If yes, do you still think it is acceptable to take down 100s of
innocent bystanders because one customer of a provider is misbehaving?

I assume here that you mean "Customer of a customer". Again, it
depends. If the customer has continual problems controlling abuse from
his customers, or you suspect that your customer is playing
"whack-a-mole", or the abuse is ongoing and/or serious and you can't
identify which of customer's customers is the cause (spoofed source
addresses, etc.) in some cases yes.

3) If yes, do you still think it is acceptable if the "misbehaving"
customer is not intentionally misbehaving - i.e. they've been hacked?

Again, it depends on the seriousness of the abuse and its affect on the
network, as well as the frequency thereof and the seriousness of the
customer in rectifying the problem. Also whether you can reasonably
isolate the abuse and disconnect only the customer's abusive customer.

3) If yes, do you still think it is acceptable if the collateral damage
(taking out 100s of innocent businesses) doesn't actually stop the spam
run / DoS attack / etc.?

If it doesn't stop it but stops your network from being a part of it,
yes. If it has no affect on it at all, then you're probably pulling the
wrong plug.

These are important question to me, and I'm surprised at the number of
people who seem to feel so very differently than I thought they would
feel - than I personally feel. Would people mind sending me private
e-mails with yes/no answers? Longer answers are welcome, but yes/no
will do.

This is IMH operational, so posting publicly. I don't think this is as
black-and-white as to warrant simple yes-no answers. There are policies
involved as well as your agreements with your peers/upstreams. If the
issue is serious enough that you risk losing your own connectivity
because you can't stem the abuse from a customer's customer, then you
may need to do so, or the end result will be that you become part of
greater collateral damage.

Using the case under discussion as an example, I am wondering why
anyone thinks taking down 100s of innocent domains is a good way to
stop a single hacked machine from doing whatever it is doing? If you
somehow think all that is worth it, take a close look at your cost /
benefit analysis. At this rate, every business on the Internet will be
out of business before we take out even a single moderately large botnet.

The present example seems to be a combination of poor communication, bad
attitude and sloppy network design from what I've seen here. It's
unclear to me exactly what GoDaddy shut down, and the only data points
we have to go on are admittedly edited conversations that took place
after the plug was pulled. What went on beforehand? Did Nectar indeed
make a good faith effort to correct the original problem? Was their
attitude the same as shown on the phone calls? How long had the problem
existed, had it happened before, and did Nectar keep an open dialogue as
to the steps they were taking to fix it? Did GoDaddy have less
intrusive options to shut down just the abuser?

I am also wondering why anyone thinks the miscreant will stop just
because the legitimate owner's domain no longer resolves? Not only is
the machine likely to continue sending spam as if nothing happened, we
aren't even "catching" the guy. I guess you could say "well, it put
pressure on his hosting provider to clean the infected machine", which
is true. I just think that's a bit silly. But maybe I'm the one who's
silly.

I think this was a case of a fake phishing website rather than outgoing
spam spew. If the domain was the target of a phish, then causing it not
to resolve would keep the phisher from reaping any benefit from the
abuse although the spam run would likely continue, at least for a while
until the phisher realizes it is in vain.

Lastly, I wonder what "average" people - people who run businesses on
hosting providers who really don't understand all this computer stuff -
think about such actions. How many 100s of people have we just
alienated for life to stop - er, NT stop - a single zombie? And how
many of their friends are going to hear over an over how the Internet
is not a real business and no one should put any faith in it?

Well, "average" people who run businesses on hosting providers" probably
should hire someone who does understand all this computer stuff to do
some due diligence on the providers they are considering. If their
prospective providers netblocks are repeatedly mentioned in SPEWS,
Spamhaus, Spamcop, and NANAE, they may want to look elsewhere.

Googling "Nectartech abuse" is interesting. As far back as July of last
year they were battling GoDaddy over spam and abuse issues. It doesn't
look like this should have been all that big of a surprise. In fact,
Nectartech's predictions in post 23 of the following thread are eerily
accurate.

Is this really a good thing?

If steps are taken to minimize collateral damage, yes. Allowing the
abuse to continue causes collateral damage to the rest of the Internet
for as long as it continues. The choice often boils down to severe
collateral damage to a few or raising the noise level and collateral
damage to the Internet as a whole. Is cutting off ten customers of an
infected customer better than allowing this customer's virus to infect
tens of thousands of random hosts on the net worth it? If you're one of
the tens of thousands, yes. If you're one of the ten customers, no.

Message
From: "Patrick W. Gilmore" <patrick (AT) ianai (DOT) net>
To: <nanog (AT) nanog (DOT) org>
Cc: "Patrick W. Gilmore" <patrick (AT) ianai (DOT) net>
Sent: Tuesday, January 17, 2006 1:09 AM
Subject: Re: GoDaddy.com shuts down entire data center?

Jan 17, 2006, at 1:32 AM, Jim Popovitch wrote:
>
>I want to say, from an outsider's perspective, that I whole heartily
>applaud GoDaddy on the actions they took []
>
There seems to be a wide split on this topic. I was wondering if people
would privately tell me yes or no on a few questions so I can understand
the issue better.

1) Do you think it is acceptable to cause any collateral damage to
innocent bystanders if it will stop network abuse?

If the damage of the persistant abuse is greater than the lost of the
innocent persons, yes.

2) If yes, do you still think it is acceptable to take down 100s of
innocent bystanders because one customer of a provider is misbehaving?

Yes I do and more than likely, so do you. If you are a common end point for
all of my users and I'm the common end point for yours, either of us has the
right to deny access to the other at any point for no reason really. Now,
should your network start flooding me or vice versa, one of us, if not both,
will toss up some filters. If either of our networks is larger than the
other and causing a dos for the other end, the effected one of us would have
no recourse but to contact the upstream of the source point and request
assistance.

3) If yes, do you still think it is acceptable if the "misbehaving"
customer is not intentionally misbehaving - i.e. they've been hacked?

Intentional or not, it doesn't negate the fact that the system has been
hacked and is now owned by someone other than the actual owner. If one of my
systems were to be hacked and I miss it, and it starts causing problems for
your network, I expect my network to be filtered. If your filters aren't
effective enough to deal with the issue, and I'm not helping you to correct
the problem, I expect you to go to my carrier to file a complaint.

3) If yes, do you still think it is acceptable if the collateral damage
(taking out 100s of innocent businesses) doesn't actually stop the spam
run / DoS attack / etc.?

There is no simple yes / no for this one. It would depend on the
circumstances of the issue.

<snip>

Using the case under discussion as an example, I am wondering why anyone
thinks taking down 100s of innocent domains is a good way to stop a
single hacked machine from doing whatever it is doing? If you somehow
think all that is worth it, take a close look at your cost / benefit
analysis. At this rate, every business on the Internet will be out of
business before we take out even a single moderately large botnet.

You can wonder why, however I, IMH, think that if more carriers would take
that stance, then the problems that we face daily would be much less severe.
Currently, there's not much to keep the big players in check when it comes
to their network. Now, imagine, what could happen if they were forced to
play by the same rules that we have to go by? If our network is causing
problems, our uplink(s) have the authority to disconnect them for that
generally. Can you see Sprint, SBC/AT&T, L3, Cogent, AL, Cox, etc having
those same rules applicable to them or be depeered from all peers and become
network dead? Now, is it feasible to do such a thing? Not usually because it
causes financial issues on both sides of the depeering. That's because the
internet that we have is used as a means of financial gain and isn't geared
for being easily segregated in the event of compromise. Yet, that's the
current mechanism for a compromised end user. The same means should be used
all the way to the NAP imo.

I am also wondering why anyone thinks the miscreant will stop just
because the legitimate owner's domain no longer resolves? Not only is
the machine likely to continue sending spam as if nothing happened, we
aren't even "catching" the guy. I guess you could say "well, it put
pressure on his hosting provider to clean the infected machine", which is
true. I just think that's a bit silly. But maybe I'm the one who's
silly.

Why should you or I be the ones responsible for catching the miscreant when
the compromised system isn't on our network? If it were, then that task
would fall to us to do so. If the threat of a delinking were over our heads,
we'd have some major incentive to find the idiot and make sure he's not on
our net anymore wouldn't we.

Lastly, I wonder what "average" people - people who run businesses on
hosting providers who really don't understand all this computer stuff -
think about such actions. How many 100s of people have we just alienated
for life to stop - er, NT stop - a single zombie? And how many of their
friends are going to hear over an over how the Internet is not a real
business and no one should put any faith in it?

Average people think email is secure.
Average people think that email is instant.
Average people think that updates and patches are a hinderance and not
necessary.
Average people think that the internet is flawless.
Average people think that their current provider is the internet.
Average people don't care what happens outside of their cable/dsl modem or
their linksys/dlink router.
Average people just want it to work and don't want to know what's behind the
scenes to make the *magic*.

Is this really a good thing?

Yes, they need to know that the net is like a shark in the water. It may not
get you today, tommorrow or never. But that doesn't mean you want to swim in
shark infested waters without taking proper precautions.