Planet Antispam

All Spammed Up: China Remains a Spam Haven Thanks To Indifferent ISPs

Fri, 03 Jul 2009 08:10:54 +0000

Researchers at the University of Alabama say almost all of the websites advertised through spam are hosted in China on servers protected by bulletproof hosting. That means that the ISPs who provide hosting to spammers and malicious domains simply don’t care and ignore abuse complaints and take down orders.

The researchers reviewed millions of spam message and found that over 69,000 unique domains hosted the websites found in the spams and of those, 70% were located in China, making it a definite spam haven.

“It is very normal that more than one-third of the domain names we see each day in spam messages come from China,” wrote Gary Warner, director of research in computer forensics at the university. “When one also considers the many ‘.com’ and ‘.ru’ domain names which are also hosted in China, the problem is much worse.”

The so-called bulletproof providers actively recruit spammers and cybercriminals, going as far as to post ads on the underground websites where they are known to socialize. These hosts ignore take down requests and abuse reports and even make IP addresses hard to trace. A Chinese domain name can be had for a mere 15 cents, which only adds to the problem.

The researchers aren’t sure all the providers hosting the spam domains are bulletproof however. They speculate that a few may simply not have the resources or understanding to deal with the problem. Curiously enough, while the Chinese government had made headlines and waves with its increasing attempts to censor the Internet in the name of fighting porn, they have had nothing to say about the spam problem. It’s not known if they are even aware that there is one!

John R. Levine: What are TLDs good for?

Fri, 03 Jul 2009 02:11:05 +0000

Yesterday I said that the original motivations for adding new TLDs were to break Verisign's monopoly on .COM, and to use domain names as directories. Competitive registrars broke the monopoly more effectively than any new domains, and the new domains that tried to be directories have failed. So what could a new TLD do?

Get rich quick: the new domains with the most registrations are .BIZ and .INFO, clones of .COM and .ORG for people who missed out the first time. Despite vigorous marketing and, for .INFO, price cutting, neither is more than a pale shadow of the original, and both are plagued with sleazy registrants. Nonetheless, we can expect a few more clones like .WEB, who will make their money from defensive trademark registrations, domain squatters, speculators, and a few suckers who think that SAUERKRAUT.WEB can be the gateway to a mail-order fortune.

Idealists: Another unpersuasive theory says that a TLD enables communities. The best example to date is .CAT for Catalonia, which is modestly successful but doesn't tell us much since Barcelona is a rich sophisticated city that would be awash in Internet content with or without a domain. On the other hand .MUSEUM is a noble failure, with only about 200 registrants, a lot of dead links, and negligible visibility. Two pleasant young men have been trying to get .BERLIN through ICANN for years, and there are other candidates like .ECO, but it's hard to see why anyone would switch from their existing domain in .DE or .ORG or whatever since they haven't for any of the community domains we have now. I've heard claims that tiny language groups in danger of dying out need their own TLD, but it seems to me that if they could raise the $185K that a TLD application costs, they'd be a lot better off hiring linguists and programmers to compile dictionaries and adapt text and web tools to work in the language.

Certification: sponsored TLDs are supposed to ensure that all of their registrants meet specific requirements, so you know that a domain in, say, .COOP is an actual co-operative. The flaw in this theory so far is that none of the sponsored TLDs so far have been in areas where there's a problem with fakes, nor do they have any process to verify that registrants remain eligible. The little poultry packer that registered CHICKEN.COOP sold out to a larger company, but nobody noticed they weren't a co-op any more until I wrote to .COOP management and told them. They thanked me and encouraged me to report any more violations I saw, so I guess I volunteered to be the compliance department. The number of registrations in .COOP is on the order of 1% of the co-ops in the world, so it appears that the other 99% of co-ops are getting along fine without a special domain.

The .PRO domain is supposed to be just for licensed doctors, lawyers, accountants and maybe other licensed professionals (the web site is a bit vague), who have to present their licenses to register, but a combination of mismanagement and financial problems have allowed in large numbers of speculators and other registrants who clearly don't meet the criteria, so it doesn't tell us anything useful. I could imagine that a .BANK domain that carefully vetted its registrants to be sure they were real banks with government banking licenses might help tell real from fake bank web sites and mail, but that certification niche seems to be taken already by green bar SSL certificates.

Branding: The new rules allow single owner domains, so we can expect Apple to get .MAC and probably other companies will register their name like .IBM or brand names. Marketers are doubtless salivating, but for regular users, it's hard to see why you'd want to be BOB.MAC and rent your identity to your computer vendor.

Non-English languages: This is the only one that has any urgency at all. China really wants .中国 in addition to .CN, and a lot of other countries with non-Roman writing would also like localized domains. ICANN has a separate process for non-ASCII TLDs, so I'll ignore them for now.

So running down this list, where's the compelling argument? Does anyone (ignoring those with vested interests) really think that more TLDs will break the .COM monopoly? That more "community" TLDs will be any more of a success than the failures to date? That anyone will use a TLD rather than a search engine as a directory?

The only unambigous beneficiary of new TLDs is ICANN, whose cash flow will increase by $185,000 per application, and all of the consultants they've hired to do the evaluations because ICANN's many highly paid staff evidently can't do it themselves. Since a lot of the new TLDs will be run by organizations with little or no experience as a registry, we can expect them to learn slowly and painfully about all the sleazy tricks that crooked registrants pull.

In sum, neither of the two classic arguments for new domains, competition and directories, have worked in the past decade, and there's no reason to think they will in the future. Other than support for non-English languages, all of the other rationales strike me as wishful thinking, not business models. So I look forward to .中国 and its ilk, but other than that, they're all going to fail, very expensively.

Box Of Meat: Jack Goldsmith in the New York Times: Defend America, One Laptop at a Time

Thu, 02 Jul 2009 18:03:58 +0000

Jack Goldsmith in the New York Times: Defend America, One Laptop at a Time: “…the private sector owns and controls most of the networks the government must protect. …the firms that build and run computer and communications networks focus on increasing profits, not protecting national security. They invest in levels of safety that satisfy their own purposes, and tend not to worry when they contribute to insecure networks that jeopardize national security. This is a classic market failure that only government leadership can correct. The tricky task is for the government to fix the problem in ways that do not stifle innovation or unduly hamper civil liberties.”

Box Of Meat: Exact Target: The Tipping Point Between Inbox and Spambox

Thu, 02 Jul 2009 17:03:26 +0000

Exact Target: The Tipping Point Between Inbox and Spambox:

“Many in our industry don’t help the matter at all because they prefer to create fear, uncertainty and doubt (FUD) around email delivery because they feel that obfuscation will serve them well and buy them customers. The outcome is confusion and distrust.”

Not sure why Chip didn’t also mention that most email marketing blogs (and twits) just repeat what other people in the industry said the day before.

Box Of Meat: Spam Wars: More on the URL Shorteners

Thu, 02 Jul 2009 16:03:19 +0000

Spam Wars: More on the URL Shorteners: “What a great way for a company to build an online brand presence—by hiding behind a URL shortener. WTF?”

Box Of Meat: ComputerWorld: Registrars under fire in domain disputes

Thu, 02 Jul 2009 15:02:59 +0000

ComputerWorld: Registrars under fire in domain disputes:

“Are domain registrars making money from cybersquatters at the expense of legitimate brands? If so, why isn’t ICANN stopping it?”

Perhaps because the fees trickle upwards, and thus ICANN is making money from this abusive behavior too?

Box Of Meat: Wired Threat Level: This Just In: Fake News Sites Are Great!

Thu, 02 Jul 2009 14:02:16 +0000

Wired Threat Level: This Just In: Fake News Sites Are Great!: “There’s something icky about the fake news ads showing up on genuine news sites like Salon, Slate and Huffington Post. … It turns out there’s a whole fake-media empire pushing the story of the massive profits to be made by gaming Google from home…. Consumers who have signed up for the $2 trial have reported being hit with surprise charges on their credit cards ranging from $70 to $80.”

John Graham-Cumming: Is James Dyson held back by the speed of sound?

Thu, 02 Jul 2009 13:41:23 +0000

I was intrigued by a story in the Daily Telegraph about a new electric motor created by Dyson. The DC motor apparently rotates at 104,000 RPM and is to be used in a portable vacuum cleaner.

The motor technology itself is switched reluctance. Essentially, the motor works by turning on and off electromagnets at just the right time to keep the rotor inside the motor spinning.

My immediate thought was 'how fast is the outside edge of the rotor moving if it's spinning at 104,000 RPM?' And shortly after that, 'how close is that to the speed of sound?'

In Electronic Weekly there's an article which states that the motor is 55.8mm across. Now, that's probably not the diameter of the rotor, but given that Dyson is attaching an impeller to the rotor anyway I'm going to take that as the diameter and work my calculations from there.

So the distance travelled in one rotation is π * 55.8mm and there are 104000 / 60 rotations per second. So, the outside is moving at 304ms^-1.

The speed of sound at sea level is 340ms^-1.

So the impeller is likely operating at near the speed of sound. I wonder if there are any nasty effects of rotating at that speed and if Dyson is close to the theoretical limit of what he can do.

There are two patent applications from Dyson that I believe cover this invention: 20070252551 and 20070278983. Neither mentions the speed of sound.

All Spammed Up: Zbot Trojan is Harvesting FTP Credentials From Major Websites

Thu, 02 Jul 2009 12:23:50 +0000

A British security vendor has discovered that the ZBot Trojan has harvested the FTP credentials of over 68,000 websites including Bank of America, the BBC, Amazon, Cisco, Monster.com and most of the major anti-spam software makers. The credentials could allow hackers to compromise legitimate sites with malicious code and drive by downloads.

To make matters worse the list of FTP credentials is stored on a server in China in plain text, making it available to anyone who stops by. Experts say they were all stolen within the past 2 weeks and most are still valid.

The ZBot Trojan has also been spotted in several email attacks masquerading as everything from a ticket confirmation from Delta Airlines to a critical update for Microsoft Outlook. If downloaded it steals personal information using a keylogger.

It’s crucial to make sure any unused FTP credentials on your website are disabled and that active ones have their passwords changed regularly. As we saw recently when hundreds of government sites in the UK were compromised and redirected visitors to internet pharmacies selling Viagra or porn sites, hackers are eager to infect legit sites. If they hit yours it could be a real nightmare for you and your customers, so stay alert and keep an eye on your servers and FTP logins!

Spam Wars Dispatches: Another Banned URL Shortener

Thu, 02 Jul 2009 07:11:02 +0000

The "The Business News" spammer who uses URL shortening services (noted here and here) has shown me another shortening service that doesn't give a crap about spam abuse reports — even though they solicit such reports directly on their home page.

I am now adding hurl.ws to my destructo spam filters. It's too bad, because on the surface they look like they want to do the right thing. Moreover, the outfit appears to be run by bluespark.co.nz, a fellow iPhone app developer (yeah, that's sort of been my day job recently). They advertise the service thusly:

Hurl is a url shortening service with a difference, ....

I guess the difference is that they turn a deaf ear to abuse complaints.

In return, my email server turns a deaf ear to any email message (from a non-whitelisted sender) whose body contains a hurl.ws URL. What's Maori for "Adios, amigos"?

Terry Zink: Bing gains, Google drops

Thu, 02 Jul 2009 01:52:21 +0000

The following is an excerpt from Investor's Business Daily:

Microsoft (MSFT), the software giant, increased its market share in U.S. Web searches to 8.23% in June from 7.81% in May, thanks to its new Bing search site, according to tracking firm StatCounter. Web search king Google (GOOG) lost share slightly, dipping to 78.48% from 78.72%.

Figures like these really annoy me. Why? Because they are using statistics inaccurately. Look at Google's "loss" of search share - a drop of 0.24%. How could they possibly measure that?

In statistics, there is always a margin of error known as the confidence interval. If you were to survey a group of users and 75% of them reported the same answer, then you cannot straight out extrapolate that to the rest of the population. If you sampled ~1000 people, then you can say that 75% of the population, +/- 4% would give the same answer. At a 95% confidence level, then you would say that you are 95% confident that between 71% - 79% of the population would give that answer.

Surveys work by doing random sampling. Yet, in order to get the responses above, we have to make sure that the margin of error is less than the difference. For example, in my above example, suppose you asked 1000 people what kind of widget they liked best and 67% of them said Widget A. Next month, you ask 1000 people the same question and and 65% of them say Widget A. Does that mean there was a drop of 2%? No, because the 2% drop is within the 4% margin of error from the previous month. You cannot be certain of anything.

In order for Google to have experienced actual market share loss, the original number had to be 78.72% +/- 0.11%, while the second number has to be 78.48% +/- 0.11%. Why? Because we have to have non-overlapping margins of error:

78.72 - 0.11 = 78.61%
78.48 + 0.11 = 78.59%

Those two do not overlap and thus we can be confident that real market share has been lost by Google. So, how many people would the survey have to interview in order to get that confidence interval? About 735,000. I somehow doubt that this surveying company actually asked that many people what their favorite search engine is (or however they did their sampling). In order for Microsoft to have gained their market share, they would need to have sampled 213,000 people. Sounds unlikely to me unless they have some automated way of culling out all of this data.

People need to know how to use statistics properly.

John R. Levine: Who needs more TLDs?

Thu, 02 Jul 2009 00:11:03 +0000

ICANN's Sydney meeting has come and gone, with the promised flood of new top-level domains claimed to be ever closer to reality. Does the world need more TLDs? Well, no.

Way back in the mid 1990s, it seemed obvious that Internet users would use the DNS as a directory, particularly once early web browsers started to add .COM to words typed in the address bar. This led to the first Internet land rush, with heavy hitters like Procter and Gamble registering diarrhea.com in 1995.

Everyone wanted to get into .COM, since that was the de-facto directory for the Internet. Network Solutions, the predecessor to Verisign, had a monopoly on registrations in .COM and that was a problem. Many people thought the solution was to add more TLDs with different monopoly registrars (often themselves.) I believe that I was the first to propose breaking the registration monopoly by splitting registries and registrars in December 1996. One of ICANN's undeniable successes is the competitive registrar market, which (as I predicted) as allowed a wide variety of sales models, and a lot of bundling of low-cost domains with web hosting and other services.

Since 1996 we've learned two things about TLDs: TLDs make a lousy directories, and users don't use the DNS for directories anyway. Several of the new TLDs introduced by ICANN since 2000 were intended to be structured as directories. The .AERO domain reserved two letter domains for airlines and three letter domains for airports, using standard industry codes, which was a clever idea, but not one that interested many airlines or airports. The .MUSEUM domain tried very hard to be a directory, with names organized both by the type of museum (metropolitan.art.museum) and the location (vam.london.museum) but that didn't work either.

A huge change in the Net since the late 1990s is that everyone uses search engines to find what they're looking for, to the extent that many non-technical users don't know the difference between the address and search boxes in their browsers. (Sometimes they'll type a search term into the address box, which keeps domain squatters in business.)

So if TLDs aren't useful as directories, what could they be useful for? We'll address the possibilities tomorrow.

Box Of Meat: Wired Threat Level: Filtering Companies Can’t Be Sued By Blacklisted Firms, Court Rules

Wed, 01 Jul 2009 22:04:12 +0000

Wired Threat Level: Filtering Companies Can’t Be Sued By Blacklisted Firms, Court Rules

Box Of Meat: Howard Rheingold on SFGate: Crap Detection 101

Wed, 01 Jul 2009 19:54:46 +0000

Howard Rheingold on SFGate: Crap Detection 101: ‘“Crap detection,” as Hemingway called it half a century ago, is more important than ever before, now that the automation of crapcasting has generated its own word: “spamming.” Unless a great many people learn the basics of online crap detection and begin applying their critical faculties en masse and very soon, I fear for the future of the Internet….’

Silent Noise: Forgot your training wheels again, spammer?

Wed, 01 Jul 2009 19:52:50 +0000

Another one who let go too early.
Fresh spam. Well relatively fresh, from one inbox yesterday (parts of it):

Subject: %SI_subj

What if you could %SI2_rnd10 your desire and %SI2_rnd11 by just %SI2_rnd12 %SI2_rnd13 step?
What if this step was %SI2_rnd14, %SI2_rnd15 and side-effect-free?

There is %SI2_rnd16 solution!
%SI2_rnd17 %SI2_rnd18 use %SI2_rnd20 to give their %SI2_rnd20 %SI2_rnd21 night fire!

If there are no %SI2_rnd22, why refusing to take one pilule before %SI2_rnd23?

%SI2_rnd24 of men did it – You can do it too!

The Internet Patrol: Bebo Spam

Wed, 01 Jul 2009 19:41:39 +0000

Well, look what the spam cat just dragged in - address book importing spam from Bebo. Why is it that these various social networking and other sites seem to simply consider the incidental spam attendant to address book importing (if they think about it at all) to be the cost of ...

Enemieslist: new pats posted - 20090701 (maintenance pats release)

Wed, 01 Jul 2009 19:41:24 +0000

39819 patterns, 11412 right anchor strings, 172127 test IPs.

Contribs from yesterday, plus more from a CBL list.txt I recently
resolved down to PTRs. This release matches 99.995% of the PTRs in
that CBL zone.

Was asked to start tracking couplets (pattern class and tech, taken
together as a sort of meta-identifier); there are no new couplets in
this release.

Download them here:

sendmail:
http://enemieslist.com/downloads/sendmail_access_db
http://enemieslist.com/downloads/rightanchors

postfix:
http://enemieslist.com/downloads/postfix_regexp_table
http://enemieslist.com/downloads/postfix_regexp_table-20090701

exim:
http://enemieslist.com/downloads/exim_hosts
http://enemieslist.com/downloads/exim_hosts-20090701

Box Of Meat: New York Times: U.S. and Russia Differ on a Treaty for Cyberspace

Wed, 01 Jul 2009 18:54:51 +0000

New York Times: U.S. and Russia Differ on a Treaty for Cyberspace: “United States officials say the disagreement over approach has hindered international law enforcement cooperation, particularly given that a significant proportion of the attacks against American government targets are coming from China and Russia.”

John Graham-Cumming: How to do customer service

Wed, 01 Jul 2009 17:25:46 +0000

I've previously complained about poor technical support that I received from Hewlett-Packard. That particular incident isn't over yet... the issue has been escalated a couple of times, HP has told me they are end-of-lifeing the product, ... I'll write that up when it comes to a resolution.

But it's not all moaning! Two companies that have provided excellent customer service recently are Apple and Bugaboo. I dealt directly with Apple myself, a friend with small children told me about the Bugaboo goodness.

First off, Apple. I own a MacBook Pro that I bought in mid-2007. Unfortunately, it suddenly started to suffer from the NVIDIA GeForce 8600M GT problem a couple of months ago. The upshot was that my machine would boot but couldn't find a display adapter (or at least it found the Intel display adapter, not the NVIDIA one).

I verified that I could ssh into the machine and ran System Profiler on the command-line. A quick search by serial number showed that my machine was susceptible to this problem and that Apple offered free service.

So, I called AppleCare. I never bought AppleCare for this machine and for this problem I didn't need it. I described my problem in detail to the technician including the steps that I'd taken to try to resolve it (including resetting the PRAM and SMC) and he did something great. He completely avoided going through any script, realized that I knew what I was talking about and immediately set the machine up for repair.

Next step was an appointment with the Genius Bar. This was the most annoying part because Apple's Concierge software is poorly designed. But once at the Genius Bar I got my appointment in about 10 minutes of the allotted time. The technician immediately verified that I had the NVIDIA problem and that I was eligible for a motherboard replacement.

While I was chatting with him I mentioned that my iPhone headphones had a fault and I wanted to buy some new ones. He asked me how long I'd had the iPhone (about 3 months) and simply went and got me a new pair, for free, just like that.

Then he told me to expect that my MacBook Pro would take about a week to repair. I left the Apple Store and went into work. That evening Apple called me to tell me the laptop was ready.

Nice.

Now Bugaboo. My friend Bill has two small kids and one of them is always in a Bugaboo Cameleon stroller. These are really high-end and expensive bits of kit. But they are very, very well made.

Now Bill's Bugaboo's brakes had developed a fault. They didn't always work and it was a minor annoyance. Little did Bill know that Bugaboo had identified this as a common fault and recalled the Cameleon.

Happily, Bill had filled out the warranty card for the stroller and sent it back when he bought it. One day a small package arrived unannounced containing a kit to fix the brakes. The kit worked perfectly.

Nice.

In both cases, Apple and Bugaboo, we were dealing with premium brands and got premium support. Apple's ability to just give me new headphones made my experience wonderful, and Bugaboo simply sending the repair kit to Bill made him a loyal customer for life (he just needs to have some more kids).

Spamresource.com: Usenet.com Gets Ass Handed To It By Court

Wed, 01 Jul 2009 17:23:35 +0000

Nate Anderson reports for ARS Technica: "A federal judge yesterday found Usenet.com liable for just about every copyright infringement claim on the books: direct infringement, inducement of infringement, contributory infringement, and (just for good measure) vicarious infringement. Not content to be loud and proud about its pro-pirate agenda, Usenet.com also resorted to stonewalling legal

All Spammed Up: Protecting Exchange Server 2007 Distribution Groups from Spam

Wed, 01 Jul 2009 12:41:08 +0000

I was discussing a spam problem with a customer recently and they mentioned to me that one of their biggest problems is spam sent to their email distribution lists. The problem had come about due to two things - firstly the email addresses for some of their distribution lists are very easy to guess (eg, the “All Staff email group has an email address of allstaff[at]company.com), and secondly there had been occasions in the past where staff exposed the email addresses by CC’ing them on emails sent outside the company.

Over time the problem has grown to the point where it is now very frustrating for their staff. They’ve asked me for some suggestions on how to fix this problem, so I presented them with these options.

Requiring Authentication for Exchange Server 2007 Distribution Groups

The default behavior for newly created distribution groups in Exchange Server 2007 is to require that all senders be authenticated, or the message is simply rejected. This is useful, however, for a vast majority of Exchange Server 2007 organisations their distribution groups existed prior to the upgrade to Exchange Server 2007. In these cases the authentication requirement is not enabled.To require authentication for a distribution group simply open the group properties, navigate to the Mail Flow Settings tab, open the Message Delivery Restrictions and then tick the box marked “Require that all senders are authenticated”.

While this solution has the desired effect of preventing spam from reaching the distribution group, it also prevents other legitimate outside email from reaching the list.

Filtering Distribution Groups by Sender

The authentication requirement will prevent legitimate outside email from reaching important distribution groups. To resolve this through the same Message Delivery Restrictions you can instead control which senders are permitted to send to the distribution group.

This method causes some extra administrative burden for the email server admins because each permitted sender must first be added as an Exchange Contact. Furthermore if you want the distribution group to receive emails from internal staff you need to ensure they are also added to the list, either directly or via a group.

Obscuring Distribution Group Email Addresses

One method that most email admins will try at least once in their career is to obscure the email address of distribution groups to make it harder to guess, or to make it impossible to send to from outside the organization. In Exchange Server 2007 this is achieved by using Email Address Policies that apply only to distribution group objects.

For example, the policy may apply a string of characters to the email address to make it harder to guess, such as allstaff_ksf2ui2[at]company.com. While this does have the effect of making it nearly impossible to guess it does nothing to prevent exposure of the email address if it were included in an email sent outside the organization.

A second technique is to use an SMTP domain that is invalid outside of the organization. For example allstaff[at]groups.company.com or allstaff[at]company.local. This has the effect of nullifying any exposure of the email address outside the organization but similar to the earlier filtering techniques it prevents legitimate outside email from reaching the group.

Implementing an Anti-Spam Solution

Although the customer was seeking a free solution once I explained each of the options above it became clear to them that these techniques would either be ineffective, require too much effort to maintain, or would prevent legitimate business use of their distribution groups.

Instead they agreed to trial an anti-spam solution, which satisfied them by preventing spam and other unwanted emails in an effective and easy to manage way, and which they ultimately purchased and are now happily getting on with their business without the constant hassle of spam.

All Spammed Up: New Malware Attack Pretends to Be a Microsoft Update

Wed, 01 Jul 2009 12:11:26 +0000

A new malware attack is lurking behind emails made to look like Outlook updates sent by Microsoft. The messages look authentic and include a link that looks like it points to update.microsoft.com but actually points to a malicious domain. If clicked the link activates a download which contains the Zbot Trojan. Zbot steals usernames, passwords and banking information and installs a rootkit that could allow a hacker access to any network the infected computer is attached to.

Zbot even contains a list of specific sites to monitor including Facebook, MySpace, Bank of America, Amazon, HSBC, Paypal, Blogger, and just about every bank you can think of. This Trojan means business. Once a user on an infected machine accesses one of the sites on the list, a built in keylogger is activated and records their information. The stolen information is then uploaded to a remote server.

Zbot has been spotted in several previous attacks. One pretended to be a notice from UPS, another a ticket confirmation from Delta Airlines and a third a notice from Western Union. The gang behind the attacks is said to be hiding out in Russia.

To protect yourself and your users, remember that common sense is a hacker’s worst enemy. They are hoping people will trust that it a real update from Microsoft even though it’s well known that Microsoft pushes their patches through on the second Tuesday of each month only and never ever sends them via email. If you get an update from anywhere other than the Microsoft Update console, chances are it’s fake. Make sure you have a policy in place regarding software installation. It’s probably best to restrict everyone but the IT department from doing any at all.

Justin Mason: User script: add my delicious search results to Google

Wed, 01 Jul 2009 09:58:59 +0000

For years now, I’ve been collecting bookmarks at delicious.com/jm — nearly 7000 of them by now. I’ve been scrupulous about tagging and describing each one, so they’re eminently searchable, too. I’ve frequently found this to be a very useful personal reference resource.

I was quite pleased to come across the Delicious Search Results on Google Greasemonkey userscript, accordingly. It intercepts Google searches, adding Delicious tag-search results at the top of the search page, and works pretty well. Unfortunately though, that searches all of delicious, not specifically my own bookmarks.

So here’s a quick hack fix to do just that:

my_delicious_search_results.user.js - My Delicious Search Results on Google

Shows tag-search results from my Delicious account on Google search pages, with links to more extensive Delicious searches. Use ‘User Script Commands‘ -> ‘Set Delicious Username‘ to specify your username.

Screenshot:

Enjoy!

Terry Zink: My take on blacklists, part 2

Wed, 01 Jul 2009 04:37:50 +0000

I'm going to attempt to summarize a blocklist without going to the article on Wikipedia. I'll be doing this straight off the top of my head.

Motivation

A blocklist is essentially a shortcut to spam filtering. Assume that you have a content filter that is doing all of the work of filtering, faithfully executing and flagging messages as spam. Everything is great except that the spam filter is doing a lot of work and occasionally, the odd spam message or two slips through. You can live with this if all you are filtering is 10,000 messages per day.

But imagine you are filtering 10 million messages per day. Suddenly bandwidth becomes an issue because most of your bandwidth is being taken up by useless data (spam). In addition, if your filter is "only" 99% effective, 100,000 spams are still leaking through to end users. If your organization has 10,000 users (a good size company), then that's about 10 spams per day to the end user.

You need a way to make this work better.

Methods

You sit down one day and start pouring through your spam samples that your end users are submitting to you. "What's this?" you say out loud to no one in particular. You observe that while the spams have no particular pattern, you do notice that they seem to be coming from a narrow set of IPs. Let's say that out of 100 messages, you see the following pattern (I'm using hypothetical IPs):

IP	Spam Count
292.144.16.11	16
292.144.16.17	15
292.144.16.19	22
292.144.16.22	18
292.144.16.27	29

"That's odd," you say again. "There seems to be a lot of IPs in that range." You do a quick WHOIS lookup of that IP and you find that the IP space is owned by the organization Canadian Pharmaspammers. "Well," you exclaim, "if these guys own those IPs, I should flat out block them all! It is very unlikely that they will ever send out anything legitimate." How do you know this? Spammers never change their spots. If a spammer sends out this much spam from these IPs, at that level of volume (100 messages randomly sampled) then you can safely conclude that they will never send out anything else.

You decide to add all five of those IPs to your own blocklist. Anything that hits your network that comes from those IPs you will reject (how this works we'll get to in a future post). You've now saved your end-users from getting spam from these IPs.

Refinements

You wipe your hands and assume the problem is solved. But it's not; users are still getting Canadian Pharmaspam! Once again, you start to grab the spam samples and looking at the connecting IP. The content is all different -- again -- but the IPs look familiar:

IP	Spam Count
292.144.16.12	19
292.144.16.14	17
292.144.16.18	18
292.144.16.21	20
292.144.16.26	27

Those IPs look similar to the IPs you previously blocklisted. You have no spam from those other IPs, but lots of spam from its sister IPs. Once again, you decide to do a WHOIS look up on that IP and notice something you didn't see before. It's listed to Canadian Pharmaspammers, but they also own the netblock 292.144.16.0/27 -- a netblock of 32 IPs. You decide to get pre-emptive; you go into your personal blocklist and remove the previous five IPs and instead insert 292.144.16.0/27. You have now listed the entire range of IPs. You only have evidence from 10 different IPs but strongly suspect that spam is coming out of all of them, and therefore you engage in a pre-emptive strike. You list the IP range, cross your fingers and hope for the best.

The next day you check your spam stats and notice something; rather than content filtering 10 million messages per day at the content filter, your upstream IP filter has cut that down to 1 million per day! Gah! That's a reduction of 90%! Your content filter is flying! Furthermore, the amount of spam complaints has gone down from 100 per day to 20 per day, a reduction of 80%. By adding these IPs to the blocklist, you have accomplished two things:

Users are seeing less spam in their inboxes because while your filters are good, there may be gaps. This blocklist fills in those gaps.
You have saved a good chunk on bandwidth and spending precious resources on less and less junk.

Those are the two basic uses of blocklists. A third would be spam filter automation and leveraging the work of others, but we'll get to that in a future post. But by and large, these impacts are immediately noticeable by everyone using the service and therefore, the use of blocklists eventually becomes indispensable if you want to run a filtering service.

Ed Falk: Spanish Prisoner scam on the rise

Wed, 01 Jul 2009 00:30:59 +0000

Just a heads-up; a variant of the Spanish Prisoner scam has been on the rise lately.

To recap: in the Spanish Prisoner scam, someone writes to you claiming to be a prisoner in a Spanish prison (the scam is said to goes back to the 1500's). If you send bail money, riches will be yours once he returns to freedom.

In the modern variant, the offer either arrives via random spam, or targeted directly to you through the compromised email account of a friend.

The latter form is the most insidious. The email actually comes from someone you know, claiming to be in dire straights of some sort or another. Typically your friend is traveling abroad, the email will say, and has been robbed of cash, credit cards, and ID. You are begged to send cash as quickly as possible so your poor friend doesn't wind up jailed as a vagrant or some other terrible thing.

If you're sharp, you might notice that your friend isn't calling you by your name. Or you might remember that your friend isn't traveling anywhere at all, and in fact you had poker night with them just last night.

If you're a little bit slow on the uptake, you might actually send some money. If that happens, expect to get requests for more (oops, too late, he got arrested for vagrancy and now needs bail money).

The requests for money will continue until you catch on or run out of money to send.

For a good account of the scam, read Gadi Evron's article Facebook Scam: I'm Stranded In London. Send Money!

So remember to be on the lookout. If you get email from a friend asking for emergency money, always double-check via some other channel. A phone call is best.

And if you're the one whose email, facebook, or other account has been used for a scam like this, be sure to contact everybody on your contacts list and warn them. Chances are, the scammer has been hitting every name on the list.

Enemieslist: The Trifecta, or, tweaking your way to glory

Tue, 30 Jun 2009 20:08:13 +0000

We have our own home-grown sendmail antispam filters here, which use a fairly broad brush to score incoming mail, but which have been remarkably effective for us for over six years.

One of the data points we check is of course whether the sending host has a generic PTR, via the enemieslist DNSBL. But we also find it useful to check the TCP fingerprint of the sending host, to see if the box on the other end is running some form of Windows - particularly certain highly vulnerable releases and patchlevels, like Windows XP Service Pack 1. We also check to see whether the message in question is in multipart/alternative format, or "HTML email", because in our experience it's rare to see spam that is in plain text format.

Each of these conditions (HTML, Windows, generic PTR) scores a fairly low spam score, because of course it's perfectly normal for mail to be in HTML format, and there are many Windows boxes running MSExchange and other legitimate Windows-based mail server software. And of course, there are many small businesses with generic addressing on their static netspace. The problem is when we see all three together.

As a default, all of our local accounts here have a spam score threshold of 4, which is sufficient to keep out the vast majority of the inbound spam - especially if the local scoring has been tweaked to give high scores to generic HELOs and low to generic PTRs - and which lets almost all normal mail traffic through. For historical reasons, the scoring is all done in integers, so we don't have the fine-tuning capabilities available in SpamAssassin, for example, where an HTML message might get a 1.7 just for containing HTML and no text part. Here, by default, HTML email scores a point, any Windows system scores a point, and any other issue is usually enough to dump it into the quarantine. A static generic PTR gets 2 points. So, the Trifecta is 4 points, enough to reject on for most accounts.

Pretty much the only time we ever have to whitelist anyone here is when the sender has hit the Trifecta outlined above. HTML-only email, sending from a Windows box, with a generic (almost always static) PTR. What's sad about this isn't that we have to make up for their IT consultants' failure to bother to request a custom PTR, or that some people run MTA software that spits out HTML-only email. No, that's pretty much par for the course in any industry without a need for a full-time IT person or team. Lawyers, galleries, non-profits, small businesses of many kinds are subject to the pressure to conform - and to pay lots of money for Exchange (when they could use free, high-performance Unix-based mail server software). And for the skills needed to install it (poorly), maintain it (poorly) and patch and upgrade it (rarely). OK, enough Unix bigotry. For now.

Some will complain that we shouldn't be blocking (or even scoring discriminately) on known "statics". The problem is that there are a lot more statically assigned IPs out there that have unfiltered access to the rest of the Internet, and are vulnerable to infection by the botnets, than there are legitimate mail servers with generic PTRs.

For example, yesterday we blocked 349 messages sent from static generics out of 8810 total rejected messages, or 4% of our total rejections, with one false positive (the message that spurred on this post). Of those, 117 were from .com or .net hosts, with the rest coming from ccTLDs we rarely have legitimate traffic from, so we can't just accept from static generics with .com or .net TLDs.

To effectively work around the infected statics problem while avoiding the occasional Trifecta-as-FP problem will take some more analysis, or, some more widespread clue among WIndows IT consultants. And we're not going to reduce our overall filter effectiveness by 4% daily just because of a once-a-quarter FP due to a lack of care on the part of someone else. So we need to tweak, and tune, our policies on this end without compromising our perimeter defenses, or adding to my quarantine watch workload.

Our system usually generates what, to our biased minds, are perfectly useful and informative error messages, especially in response to particular problems. The problem with the Trifecta is that we're blocking based on a score, not a specific set of problems, so the error looks like this:

554 5.7.1 HISCORE Contact postmaster@hesketh.net if this is in error, but your message was rejected as spam; it simply failed too many tests. (threshold: 4; score: 4)

There's a token (for our stats), immediately followed by a contact email address that is more or less unfiltered, a rationale, and a score/threshold. The problem is that many Exchange servers either truncate the error message, rendering it less useful, or explain that the remote system did not provide a reason - often including the complete error message beneath! - which most people don't bother to read. So we get phone calls to the effect that our system is blocking their mail. Which it is, and in many cases these are actual false positives. So we whitelist their IP address, and they can send again. (Incidentally, of the 349 messages we rejected, six had a 4/4 threshold/score; one of those was the false positive. Two had a 4/5, two had a 4/6, three had a 4/7. So, one way to deal with this is to raise our default threshold to 5, thereby letting in 7 more spams a day in order to prevent a quarterly FP. This on a system where userbase-wide we see about 3 or 4 spams/day make it through the filters, and maybe a couple 419 scams and phishing scams. So, a difficult choice - how tolerant do we become, and how low do we sink in order to accommodate these arguably at-fault systems?)

What's even more annoying is that once we've whitelisted the sending IP address of one of these poor victims, they'll go home and try to send from Outlook Web Access, which many IT consultants set up on yet another IP address, also with a generic static PTR. So we go through the whole rigamarole again, only this time with their OWA IP address.

The real problem here is two-fold: the failure of IT consultants to have even the most basic understanding of the nature of deliverability and its relationship to the generic PTR question, and the continuing acceptance of such a low standard of compliance with email community norms. (And yes, there's a third factor, namely, my reluctance to raise the default spam score threshold just to accommodate these edge cases.)

So let me close with a plea to any IT consultant tasked with setting up a Windows-based mail system: please, for the love of all that is good and holy, ask your customers' ISPs for custom reverse DNS for any system legitimately sending mail. We'll tolerate your HTML-only email, and your choice of Windows, if you'll do your part and signal to us with a custom PTR that this is a system that is intended to send mail, rather than an infected end-user system or NAT or insecure LAN.

Enemieslist: new pats posted - 20090630 (maintenance pats release)

Tue, 30 Jun 2009 19:26:58 +0000

39710 patterns, 11430 right anchor strings, 171622 test IPs.

Contribs from yesterday, plus more from a CBL list.txt I recently
resolved down to PTRs. This release matches 99.995% of the PTRs in
that CBL zone.

Was asked to start tracking couplets (pattern class and tech, taken
together as a sort of meta-identifier); there are no new couplets in
this release.

Download them here:

sendmail:
http://enemieslist.com/downloads/sendmail_access_db
http://enemieslist.com/downloads/rightanchors

postfix:
http://enemieslist.com/downloads/postfix_regexp_table
http://enemieslist.com/downloads/postfix_regexp_table-20090630

exim:
http://enemieslist.com/downloads/exim_hosts
http://enemieslist.com/downloads/exim_hosts-20090630

Spam Wars Dispatches: More on the URL Shorteners

Tue, 30 Jun 2009 16:15:56 +0000

Not all URL shorteners are created equal when it comes to handling abuse complaints. Yesterday's flood continues. I went back to see how my abuse reports faired. Of the services I contacted, the only one that seems truly diligent about stomping out spam abuse of their service is is.gd. Four gold stars for them!

The one that looks to be the most problematic is kl.am, which appears to be run by a Tennessee "online marketing" firm called Sitening LLC. Unlike the responsive shorteners, kl.am does not have an abuse reporting link on their main page...or anywhere. Moreover, the main page is titled:

Shorten URL with URL Shortener for Internet Marketers

In other words, they seem to be encouraging the use of URL shorteners by commercial emailers. What a great way for a company to build an online brand presence—by hiding behind a URL shortener. WTF?

OTOH, it makes it easy for me to handle any company that takes advantage of this shortening service for spam purposes. From hereon, any email message body that contains a kl.am URL arriving from a non-whitelisted address goes straight to dev/null. See y'all!

(Tinyurl may be next.)

John Graham-Cumming: The 1944 US Presidential Election was fraudulent

Tue, 30 Jun 2009 15:04:35 +0000

OK, it wasn't really, but I thought I'd run the Scacco/Beber analysis on that election and see what it comes up with. Guess what.

If you look at the non-adjacent, non-repeated digits in the last two places in the votes counts by state for Roosevelt and Dewey you discover that 59.38% of the votes are non-adjacent, non-repeated. If the numbers were truly random you'd expect 70%. That's way worse than the 62.07% in the Iranian election.

If you then do the old Z-Test you get a Z value of -2.49 with a p-value of 0.013. That's well below the 0.05 critical value so you can reject the null hypothesis. The final digits are not random.

Is this fraud?

Is there any suggestion that the state-level numbers in the 1944 US election were invented by people?

If not, how can anyone claim that this test indicates fraud in the Iranian election?

Now run the other bit of their test looking at the frequencies of the last digit. You get 'too many' 7s (expected 10%, got 16%) and 'too few' 1s (expected 10%, got 5%).

I'm telling you, man, what's the chance of that happening, and the non-adjacent, non-repeating digits thing? (It's about 0.17% according to simulation) I mean, come on, that's gotta be fraud.

Oh, wait, it's not.

All Spammed Up: Phishing Down Under

Tue, 30 Jun 2009 13:36:11 +0000

The Sydney Morning Herald reported yesterday that a new scam is making the rounds in the land down under. A perpetrator of a phishing scam has created an email scam, claiming to be the Australian Tax Office (ATO). The email promises Aussie taxpayers a $250 bonus with their tax return, and sends them to an online form that asks for their tax information, along with their bank account data.

The web site containing the form then asks the victim to mail a printed copy of the form to an address. The print-and-send is just a ruse though, the data is actually captured through a hack when the victim presses the “print” button. The email, like many such scams, attempts to create a sense of false security, by claiming the print-and-send routine is being done for the victim’s safety.

Officials still have not been able to trace the source of the fraudulent email sender, who is using a bot network to send the emails. The ATO recommends that people delete emails like this immediately, and advises that they do not ask people to provide personal information by email. The same holds true for most, if not all, tax collecting agencies in other countries.

John Graham-Cumming: The Scacco/Beber analysis of the Iranian election is bogus

Tue, 30 Jun 2009 12:27:06 +0000

OK, I wasn't going to write another blog entry about the 2009 Iranian election, but the article in the Washington Post that supposedly gives statistical evidence for vote fraud just won't die in the blogosphere and just got a boost from a tweet by Tim O'Reilly.

The trouble is the analysis is bogus.

The authors propose a simple hypothesis: the last and second-to-last digits of vote counts should be random. In statistical terms this is often called uniformly distributed, which just means that they are each equally likely. So you'd expect to see 10% 0s, 10% 1s, 10% 2s, and so on.

Of course, you only expect to see that if you had an infinite number of vote counts because the point about random processes is that they only 'even out' to the expected probabilities in the long run. So if you've got a short run of numbers you have to be careful because they won't actually be exactly uniform.

To confirm that try tossing a coin six times. Did it come up with exactly 3 heads and 3 tails? Probably not, but that doesn't mean it's unfair.

Now, given some run of numbers (vote counts for example), the right thing to do is ask the statistical question "Could these numbers have occurred from a random process?" If they couldn't then you can go looking for some other reason (e.g. fraud).

The question "Could these numbers have occurred from a random process?" is given the ugly name the 'null hypothesis' by stats-heads. That just means that thing you are testing.

More concretely, the Scacco/Beber null hypothesis is "the last and second-to-last digits in the vote counts are random". What you want to know is with what confidence can you reject this, and for Scacco/Beber rejecting means fraud.

Now, what you don't do is go count the last and second-to-last digits, look for some that have counts that deviate from what you expect (the exactly 10% figure) and then try to work out how often that happens. That's like tossing a coin a few times, noticing that heads has come up more than 50% of the time and then starting to think the coin is biased.

Unfortunately, that's essentially what Scacco/Beber did. They picked on two numbers that lay outside their expected value and went off to calculate how frequently that would occur. That's cherrypicking the data.

What you do do is apply a chi-square test to figure out whether the numbers you are seeing could have been generated by a random process. And you use that test because it gives you the probability with which you can reject your null hypothesis.

To prevent you, dear reader, from having to run the test I've done it for you. I took their data and wrote a little program to do the calculation against the last and second-to-last digits. Here's the program:


use strict;
use warnings;

use Text::CSV;
my $csv = Text::CSV->new();

my %la;
my %sl;

foreach my $i (0..9) {
  $la{$i} = 0;
  $sl{$i} = 0;
}

my $count = 0;

open I, "<i.csv";
while (<I>) {
  chomp;
  $csv->parse($_);
  my @cols = $csv->fields();
  for my $i (@cols[1..4]) {
    my @d = reverse split( //, $i );
    $la{$d[0]}++;
    $sl{$d[1]}++;
    $count++;
  }
}
close I;

print "Count: $count\n";

my $e = $count/10;

my $slchi = 0;
my $lachi = 0;

foreach my $i (0..9) {
  print "$i,$e,$sl{$i},$la{$i}\n";

  $slchi += ( $sl{$i} - $e ) * ( $sl{$i} - $e ) / $e;
  $lachi += ( $la{$i} - $e ) * ( $la{$i} - $e ) / $e;
}

print "slchi: $slchi\n";
print "lachi: $lachi\n";

Here's a little CSV table that you can steal to do your own analysis:


Digit,Expected Count,Second-to-last Count,Last Count
0,11.6,10,9
1,11.6,9,11
2,11.6,15,8
3,11.6,6,9
4,11.6,11,10
5,11.6,11,5
6,11.6,14,14
7,11.6,18,20
8,11.6,13,17
9,11.6,9,13

And true enough I get the same figures as Scacco/Beber. The number 7 does occur 17% of the time in the last digit, and the number 5 only occurs 4% of the time. But, I don't care. What I want to know is, is the null hypothesis wrong. Could these results have occurred from a random process? And with what likelihood.

So here's where I avoid staring at the numbers (which can get to be borderline numerology) and do the chi-square test.

For the last digit the magic chi-square number is (drum roll, please): 15.55 and for the second-to-last digit it's 9.33. Then I go to my chi-square table and I look at the row for 9 degrees of freedom (that corresponds to the 10 possible digits; if you want to know why it's 9 and not 10 go read up on the subject) and I see that the critical value is 16.92.

If either of my numbers exceeded 16.92 then I'd have high confidence (greater than 95%) that the digit counts were not random. But neither do. I cannot with confidence reject the null hypothesis, I cannot with confidence say that these numbers are not random, and I cannot with confidence, therefore, conclude that the vote counts are fraudulent.

What this means is, is that there is no 'statistically significant' difference between the Iranian results and randomness. So, what we learn is that this statistical analysis tells us nothing.

It doesn't mean that the numbers weren't fiddled, it just means that we haven't found evidence fiddling.

PS In the notes added to their annotated version of the article Scacco/Beber mention that they did the chi-square test and got a p-value of 0.077. This is below the 'statistical significance' cut off of 0.05 and so their results are (as I find) not statistically significant.

To put 0.077 in context it means that there's a 7.7% chance that the digits are random. Sounds small but 7.7 is approximately 8 in 100 or 4 in 50 or 2 in 25 or ... 1 in 12.5. i.e. in 1 in every 12.5 fair elections we shouldn't be surprised to see the sort of figures we saw in Iran. That's pretty often! That's why chi-square tells us not to find non-randomness in the Iranian results.

30 June 2009 Update: I've removed that paragraph because that interpretation of the p-value is arguably inaccurate and if you are a statistician you'd probably shout at me about it. Doesn't change the fact that the data says the Iranian result is not statistically significant; it just says that my attempt to do a 'layman's version' is faulty.

To come up with better layman's version I ran a little simulation to find out how often you'd expect to see one digit occurring more than 17% of the time with another occurring less than 4% of the time (as in the Iranian election). The answer is about 1.48% of the time, or in about 1 in 67 fair elections.

John Graham-Cumming: The Iranian Election Detector

Tue, 30 Jun 2009 11:36:38 +0000

OK, I thought I was done criticizing the Washington Post Op-Ed about how statistics leave 'little room for reasonable doubt' that the Iranian election was fraudulent. But then Hannah Devlin at The Times did her own analysis and it got me thinking about the errors in that article again.

Firstly, my previous post talks about the right way to determine whether the digits are random or not, I'm not going to go over that again, but I am going to go back over some of the actual figures that are presented in the article.

So begin with this quote:

But that's not all. Psychologists have also found that humans have trouble generating non-adjacent digits (such as 64 or 17, as opposed to 23) as frequently as one would expect in a sequence of random numbers. To check for deviations of this type, we examined the pairs of last and second-to-last digits in Iran's vote counts. On average, if the results had not been manipulated, 70 percent of these pairs should consist of distinct, non-adjacent digits.

Not so in the data from Iran: Only 62 percent of the pairs contain non-adjacent digits. This may not sound so different from 70 percent, but the probability that a fair election would produce a difference this large is less than 4.2 percent.

And there's a footnote:

This is a corollary of the fact that last digits should occur with equal frequency. For an arbitrary second-to-last numeral, there are seven out of ten equally likely last digits that will produce a non-adjacent pair. Note that we treat both 09 and 10 as adjacent.

Firstly, I believe they mean to say that they treat 09 and 90 as adjacent (not 09 and 10). That means that for any number there are two possible adjacent digits out of a ten, in other words 20% of digit pairs are adjacent, so 80% of digit pairs are non-adjacent.

In their article they say 70% 'distinct, non-adjacent'. OK, so their definition of non-adjacent means that you need to exclude repeats as well (so 23, 32 and 33 are all to be excluded).

They then present the argument that a figure of 62% or less will only happen in 4.2% of fair elections. Nowhere do they explain how they derived this figure, so I decided to run a simulation. (Hannah Devlin argues that this number is incorrect in her article, worth a read)

I ran a simulation of 1,000,000 elections that generate 116 counts of votes and I looked at the adjacent pairs of numbers in the vote counts and then I calculated the percentage of fair elections that would result in the same 62% or less as seen in the Iranian election. The figure is 2.66%. 2.66% of fair elections would produce the result (or 'worse') seen in Iran.

The difference, 4.2% vs 2.66%, comes about because the figure that they must have used is not 62%, but 62.07%. That is the actual number, to two decimal places, that comes from analyzing the digit distribution in the Iranian election results.

(Email me if you want my source code)

So, what does that tell you? That in almost 3 in 100 fair elections we would have seen the result in Iran. Or if you use their numbers 4 in 100. Either way that's pretty darn often. In the 20th century there were 26 general elections in the UK. Given their 4/100 number is 1/25 we shouldn't be at all surprised if one of those general elections looked fraudulent!

Now, we expect that the percentage of non-adjacent digits is normally distributed. And, in fact, my little simulation shows a nice little normal distribution centered on 70 with a standard deviation of 4.27.

So, we've got normally distributed data, a mean and a standard deviation and a sample (62.07%). Hey, time for a Z-test!

For this situation the Z value is -1.86 which yields a p-value of 0.063 for a two-tailed test (I'm doing two-tailed here because what I'm interested in is the deviation away from the mean, not the specific direction it went in). That's above the 0.05 value typically used for statistical significance and so we can't from this sample determine that there's statistical significance in the 62.07% figure.

So, I'd say that based on the figures given I can't find statistical significance. So I don't learn anything from that about the Iranian election.

Given that the Z-test on their 'non-adjacent, non-repeated' digits test doesn't find statistical significance, and my previous piece showed that the chi-squared test on the other claim in their paper didn't find statistical significance (that was on the randomness of the last two digits).

You might be scratching your head wondering how the authors made the claim that this was definitely fraud (their words: 'But taken together, they leave very little room for reasonable doubt.')

Well, what they do is take the probability of seeing the 62% or less number in a fair election (4.2%) and multiply it by the probability of seeing the specific variance they see in the digits 7 and 5 in a fair election (4%) to come up with 1.4% likelihood of this happening in a fair election:

More specifically, the probability is .0014 that a fair election (with 116 vote counts) has the characteristics that (a) 62% or fewer of last and second-to-last digits are non-adjacent, and (b) has at least one numeral occurring in 17% or more of last digits and another numeral occurring in 4% or fewer of last digits.

That's a very specific test. In fact, it's so specific that I'm going to name it the "Iranian Election Detector". It's a test that's been crafted from the data in the Iranian election results, it's not the test that they started with (which is all about randomness of digits, and adjacency).

So, let's accept their 1.4% figure and delve into it... that's 1.4 in 100 elections. That's roughly 1 in 71. So, they are saying that their test would give a false positive in 1 in 71 elections.

How is that 'leaving little room for reasonable doubt'?

Enemieslist: new pats posted - 20090629 (maintenance pats release)

Mon, 29 Jun 2009 22:33:54 +0000

39669 patterns, 11429 right anchor strings, 171310 test IPs.

Contribs from the past few days, plus more from a CBL list.txt I
recently resolved down to PTRs. This release matches 99.995% of the PTRs
in that CBL zone.

Was asked to start tracking couplets (pattern class and tech, taken
together as a sort of meta-identifier); there are no new couplets in
this release.

Download them here:

sendmail:
http://enemieslist.com/downloads/sendmail_access_db
http://enemieslist.com/downloads/rightanchors

postfix:
http://enemieslist.com/downloads/postfix_regexp_table
http://enemieslist.com/downloads/postfix_regexp_table-20090629

exim:
http://enemieslist.com/downloads/exim_hosts
http://enemieslist.com/downloads/exim_hosts-20090629

Terry Zink: The Council of Elrond

Mon, 29 Jun 2009 20:11:18 +0000

A couple of weeks ago, the Financial Times ran an article entitled "Secret War on Web Crooks Revealed." Here's an excerpt:

The people who run the world's internet systems are a rather secretive bunch. Three times a year, senior technical officers from companies such as Google, Yahoo, AT&T, Comcast and Verizon meet to discuss ways of stopping the internet from being swamped by rising levels of spam, viruses and hacking attacks by organised criminals. They do not generally like discussing these meetings. "Some people might get nervous if they knew all the things we talked about," said Michael O'Reirdan, chairman of the Messaging Anti-Abuse Working Group (MAAWG). "It’s our job to make the internet safe, but we don't want to put people off using the web." They are also worried about being targeted by the cyber-criminals they are trying to thwart.

Indeed, it is a secretive group. It's kind of like the Stonecutters. Things are discussed there and the idea is to come to a consensus and make recommendations about how to make the Internet safer and less a haven for (un)common criminals.

Now, not having been to these latest meetings, I don't know for certain what goes on. But I have been to other, non-MAAWG meetings and I certainly know what goes on there. I have also been to a lot of cross-group meetings at Microsoft and I'm fairly certain that the types of meetings at Microsoft probably are not too much different than MAAWG. So allow me to speculate a bit.

MAAWG is attended by hundreds of well-intentioned and well-meaning people. They want to get rid of the dark evil that are spammers, malvertisers, virus writers, and all of their ilk. Yet, coming to a consensus on all these things is very difficult. People from industry have competing interests from people in research groups, or people in government, or people in the IETF or ARIN. And when people with competing interests try to come to a resolution about how best to proceed, sometimes it can take a while to make any progress. Of course, MAAWG has made very great strides in mitigating email abuse.

And that brings me to another point. This past weekend I was watching The Fellowship of the Ring. I got to the scene in Rivendell after Frodo has brought the ring there, and Elrond calls a meeting with representatives from Gondor, the Elves and the Dwarves. The Ring is presented to everyone in attendance and there is a general agreement that the Ring must be destroyed because it is so evil. I view this like MAAWG - everyone in attendance there agrees that spammers are evil and must be stopped (maybe not destroyed).

But at the Council of Elrond, everyone disagrees about the best way to dispose of the ring. Dwarves don't want Elves to carry the Ring, Elves don't trust Dwarves and the race of Men want to use it as a weapon against the forces of Mordor. I kind of see this as anti-spam fighters engaging in dubious tactics to shut down spammers (such as breaking into their servers and stealing data or deliberately inflicting sabotage). Arguments ensue and nobody gets anywhere. This is kind of like competing solutions and standards fighting it out in the real world, and in the meantime spammers are still sending their payload.

Eventually, Frodo speaks up and announces he will take the ring, though he does not know the way. Everyone looks at him and though in disbelief, they agree that the ring should go with the Hobbit. An agreement has been reached. This is like MAAWG, or CAUCE, or whoever finally agreeing to some standard way of doing things (like DKIM or SPF, or ARF format for reporting abusive mail, and so forth). Progress is being made and the enemy's progress has been impeded.

Maybe it's not the best analogy, but it's the one that floated into my mind when I watched that scene.

BTW, I'm no Frodo. I think I identify more with Boromir.

Spam Wars Dispatches: URL Shorteners in Spam

Mon, 29 Jun 2009 16:30:42 +0000

Some "business opportunity" spammer has been flooding the intertubes with brief messages that use just about every URL shortening service on the planet, including several I had never before heard. Here is a sample of the source code of one of them sent from a pwned PC in Brazil (identifying bits [removed] or xx'd):

Received: from 189-19-xx-xxx.dsl.telesp.net.br (189-19-xx-xxx.dsl.telesp.net.br [189.19.xx.xxx]) by dannyg.com (8.12.11.20060614) id n5TEwPrW040161 for <[removed]@dannyg.com>; Mon, 29 Jun 2009 08:58:26 -0600 (MDT)
Message-ID: <4A48D688.1018475@{$FROMDOMAIN$}>
Date: Mon, 29 Jun 2009 14:58:16 GMT From: Stephanie <StephanieLoyd36@{$FROMDOMAIN$}>
User-Agent: Thunderbird 2.0.0.12 (Windows/20080213)
MIME-Version: 1.0
To: <[removed]@dannyg.com>
Subject: Online Jobs : The Next Goldrush?
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
X-UIDL: 2&j!!~ai"!RaD!!)[8"!

Someone wants to share this news article with you:
http://xx.xx/11Zk

User Comment:
im pretty motivated after seeing this..what do you think?

Source: The Business News

I've filed half a dozen abuse complaints to the URL shortening services in the last 12 hours in the hope that the offending URLs will be shut down quickly. Of the shortening service Terms of Service that I've read, none of them permit using their domains as spam destination halfway houses.

Note, by the way, how the botnet software fails to mail merge a bogus domain name into the Message-ID: and From: header field placeholders.

The shortened URLs lead to a domain that claims to be registered by someone in China and has been alive for about a week. Ah, if only China would shift its internet blockage infrastructure into reverse....

So, I guess I'll keep playing whack-a-mole until one of us gets bored. Hint:it won't be me.

Ed Falk: Spammer Ronnie Scelson arrested, charged with molesting teenage girl

Mon, 29 Jun 2009 11:19:51 +0000

Long-time spammer and all-around scuzzball Ronnie Scelson has been arrested in Slidell, LA and charged with molestation of a juvenile, forcible rape, possession of marijuana, possession of drug paraphernalia and possession of a weapon while in possession of narcotics. See WWLTV news article Man arrested after allegedly cuffing teen to chair, molesting her for the full story.

Scelson, you may remember, is known for sending out spam capitalizing on the 9/11 attacks (see Mercury News), then defending himself in this usenet thread.

Update: The Times Picayune has more on the story, adding a rape accusation involving a 15-year-old-girl.

Spamresource.com: Ask Al: Help prevent a bad thing!

Mon, 29 Jun 2009 09:51:01 +0000

Terry writes, "My manager wants to take all of our emails addresses in our "pending" list (ones that haven't clicked the link for the double opt-in confirmation) and convert all 10,000+ of those addresses to active and start mailing them. My problem is no matter what I say he feels that he has the right to do it. Is there anyway you can help convince him that this is bad of business, will

Enemieslist: Links Roundup

Mon, 29 Jun 2009 00:00:48 +0000

Beckstrom to head ICANN
I wonder if it will change anything?
Spam Crisis in China
Spammers Poisoning Twitter Trending Topics to Spread Spam and Malware
Man who led spam scam pleads guilty in Detroit
DoJ scores largest ever CAN-SPAM prosecution
Yet Another Fake Microsoft Update Email Scam Making the Rounds
SORBS Blacklist on the Ropes
The Catch-22 of Email List Building
Man arrested after allegedly cuffing teen to chair, molesting her
One of the biggest scumbags in the history of spam gets his just desserts
Former spam king booked with molesting one juvenile, raping another in St. Tammany
Microsoft Co-Founder Paul Allen Starts New Email Filtering Company
Because Outlook sucks so bad it takes a startup to add basic filtering capabilities?
IP relisted despite no more mail being sent
Mickey Chandler digs into Barracuda's methods

Spam Wars Dispatches: Item Followups

Sun, 28 Jun 2009 19:17:34 +0000

I'm going to combine updates of two different items in this posting. One is kinda funny, the other not at all. I'll deal with the unfunny one first.

In yesterday's post, I talked about a malware-looking ecard spam message that led to a medz spamming page. The campaign continues, but the URLs are now leading to an executable PC file (.exe) that is pure malware.

What interests me most about this is that for the past few years, I've seen plenty of evidence that the originators of many malware lures and the so-called Canadian Pharmacy medz (and other) spam are one in the same. "They" mail to the same lists (which include some of my spamtrap addresses), and there is a similarity to their campaign tactics. I think the medz link in yesterday's email was a glitch in their system, and it only added more to the argument that this medz/sex/knockoffs spam gang is actively involved in building botnets and stealing private information (e.g., trojans that steal password credentials).

I'd like to think that if those who buy from the spammers knew they were funding malware development and distribution activity, they'd think twice. But that's like saying a heroin addict who learns where poppy plants are grown would care about funding the Taliban.

For part two of this update, I remind you of the posting about a 419er who exposes 400 email addresses in his "You've won an award!" spam. I just saw a spam message from a 419er who indavertently acknowledges he's not smart enough to figure out how to disguise recipient email addresses as blind copies (BCC). But he is aware that the To: addresses are open for viewing:

From: MARSHALL CHI
Subject: HI

the nigeria government is given $35,million us$ contract payment for
2010 africa world cup to 80 lucky people, all the 80 emails are will
shown please cross check to see if you can see your email if you do
please kindly fill this form below.

1, your full name
2, your phone number
3, your country
4, your sex
5, your age
6, your home address
7, your occupation
8, your international passport

please reply to this email address below

[removed]@hotmail.com

Isn't it odd that the 80 lucky people all have email addresses that start with the same two letters? This guy has a way to go before he understands how to send his blocks of spam to a randomized list of rented addresses if he intends to expose them. Oh, and he also needs to learn how to count because the contiguous block of addresses in the message I received contained 90 addresses, not 80.

Bad 419er! Go sit in the corner.

Spamnation: Bind their kings in chains

Sun, 28 Jun 2009 04:39:10 +0000

June looks like being a bad month for some of the big names in the world of spam. First to hit the news was Sanford Wallace, who may face criminal charges for spamming Facebook in defiance of a court-ordered injunction. Then, on Monday this week, Alan Ralsky pleaded guilty to charges of wire fraud, money laundering and violations of the CAN-SPAM Act in connection with a stock spam scheme. His son-in-law and three others also face heavy fines and possible jail sentences for their part in the scheme.

Finally, Tuesday saw the arrest of 'Cajun Spam King' Ronnie Scelson, who faces charges related to the forcible rape of one teenage girl and the molestation of a second. Scelson may also be charged with drug possession, while examination of computers seized from his home may lead to additional charges.

Spam Wars Dispatches: How to Piss Off Your Potential Customers

Sat, 27 Jun 2009 21:36:21 +0000

If you've ever read much about marketing, you've probably heard the expression "underpromise and overdeliver" as a way to impress customers with more than they thought they'd receive. I just saw a spam message that exercises a corollary to that old expression: "overpromise and deliver squat."

Here's the message:

Subject: You've received a greeting ecard
Good day.
You have received an eCard

To pick up your eCard, choose from any of the following options:
Click on the following link (or copy & paste it into your web browser):

http://[removed]view.com/

Your card will be aviailable for pick-up beginning for the next 30 days.
Please be sure to view your eCard before the days are up!

We hope you enjoy you eCard.

Thank You!

If you've been monitoring the spam business as long as I have, the first conclusion jump is that the destination URL is a drive-by malware installation web site. Seen it a gazillion times before.

But if you're an unsuspecting email user, you likely believe that there is an ecard from an unknown admirer at the end of that link. Oh, goody goody!

You click.

And what do you see? Why none other than the spam defender's worst nightmare:

So much for your secret admirer. Buy some fake Viagra instead.

I may not be the world's best businessperson, but I know enough not to aggravate your customers when they come through the door and make them feel like stupid asses for having entered.

Terry Zink: My take on blacklists - what's known, what's misunderstood and what's too good to be true

Sat, 27 Jun 2009 01:54:48 +0000

One of the stories that is circulating around the Internet this week is the announced imminent closure of the SORBS blocklist. Al Iverson of SpamResource has a good summary of it. SORBS has had its share of criticism in the past, however. From Wikipedia:

Spam database removal procedure

In order for IP addresses that have spammed in the past to be removed from the spam database, SORBS requires what it calls a "fine" in the form of a US$50 donation to a registered charity. This donation is only required for deletions from the spam database that have not expired automatically, and it is waived both for IP addresses that have been reallocated elsewhere or if the ISP implements outbound content-based spam countermeasures.However, because of these requirements, SORBS's removal procedure has been compared to extortion, but SORBS says it is not.

In the antispam community, this particular blocklist has had its detractors who say that dealing with the list has been a nightmare. On the opposite end, others say that the list has been nothing but professional with them.

I won't comment or give my particular opinion on SORBS. Rather, the announced closure of the list has prompted me to finally start a small mini-series on a topic that has been floating about in my head for several months now: what does it take to set up and run an RBL? And, more importantly, what does it take to maintain an RBL?

The goal of this series is to examine what goes on behind the scenes of compiling and maintaining an RBL. We've run a private one for three or four years now and maintaining it has been no picnic. Things break down, disks run out of space and the people who wrote the original scripts (in three days with tons of bugs) move on. Thus, I suppose one could call this the Complete Guide to Running a Blocklist in the Real World.

Remember, I deal with reality. Because we run a service, we know who our blocklists affect and that it impedes real mail flow. We also deal actual complaints and our policies are affect accordingly. It should be a good series.

Enemieslist: Poor choices in automatic / registration-based naming

Fri, 26 Jun 2009 19:22:03 +0000

One of my favorite memories from the past few years of scanning networks' PTRs is the day I ran into some spam from uncg.edu, the University of North Carolina at Greensboro. Seems the naming scheme for their dynamic networks was something along the lines of e.g.:

notceperryuncgedu.uncg.edu [152.13.115.41]

I saw a few more like this (munged to protect the poor folks), all with hostnames ending in uncgedu, and gave my usual sigh of resignation, but then I saw one that banished all doubt from my mind as to what was really going on:

notpantha91aolcom.uncg.edu [152.13.116.78]

Yes, that's right, they were using the end user's email address, stripping the dots and the @, and making that the hostname. Now, none of these example IPs resolves to anything, and I'm pretty confident they're no longer doing that. But it really freaked me out that anyone could be so lackadaisical with their end users' personal information, especially their email addresses.

Enemieslist: Principles

Fri, 26 Jun 2009 16:56:20 +0000

If I could sum up one lesson that I'd like for anyone who reads these pages to take away it would be this:

The act of naming is personal, but with communal aftereffects.

Another way of thinking of this, if you don't mind a foray into religious studies (and if you do, skip to the next paragraph, it's okay), is that naming is perhaps the most sacred act an individual may perform, being an act of participation in a divinity and community of language. But bear in mind that religions, by definition, aren't mystical - the very term "religion" defines a community of belief and a shared language for discussing those beliefs - so naming participates in both a very personal act of recognizing and externalizing your perceptions, and a very social act because the name becomes available for others to use, interpret, and acknowledge.

While it may be perfectly fine for me to name my child "Beeblebrox", because that's what I always think of him as, the name will be used by other people, too. That's one reason why despite the fact that a parent may always have a nickname for a child, they still give the child a respectable name (unless the parent in question is Frank Zappa, of course). That's just a metaphor for CNAME and PTR, BTW. And may well have nothing to do with what's actually in /etc/hostname, either - the PTR is for external recognition, CNAME for alternate use. Oh, and you're not Frank Zappa. He could get away with it. You can't.

Names, once given, convey information, and do so beyond the local context. Just because I call a server "skynyrd" because (personal reason) it had one incident where it crashed unexpectedly doesn't mean I shouldn't name it so that (public, community-oriented reason) it may be recognized for what it is beyond my network, in this case a database and Web server. In the case of an IP dynamically assigned to residential cable users via DHCP, don't assume that just because your whois SWIP for that netblock has a memo or note to that effect, that the PTRs don't need to - why not keep the name itself as the locus of such information? You're more likely to change the PTRs than you are to remember to change the note in a whois record, once those IPs are reallocated as statically assigned commercial DSL. In other words:

Maximize the information associated with a name, and keep it closest to the individual unit to which it is associated.

Think of the concept of identification. In its most basic of definitions, identification is where one thing is the same as another - in this case, a name refers to an object, so the name may be used in place of the object - they are the same thing for the purposes of the particular context. In slightly more scary contexts, you might be traveling and have someone demand to see your identification, in other words, the papers that certify your name and enable strangers to confirm that the name (and perhaps picture, fingerprints, and other biometrics) matches the body.

When naming servers (or dialup ranges, or NAT pools, etc.) think of the context in which the names will be evaluated - it will be a stranger, probably not thrilled to be evaluating your host(s) at all, probably considering them somewhat of a threat, and so forth. Worst case, it will be a stranger's leave-behind rules for evaluating the same, and there will be all the personality of an automatic teller machine involved. When naming, put your best, most formal foot forward, and don't crack jokes in line at the bomb screening.

Names are detachable containers of information; don't assume the local context and assumptions and codes will survive translation to a new context.

Finally, names differentiate one object from another. If I call one name server itchy and another scratchy, I can tell the difference between the two. If I name every last one of my end user PTRs tm.net.my, or beamcablesystem.in, the names fail the differentiation test. Put another way, there is an inherent distrust in empty, generic labels - think of "Agent Smith" from the Matrix movies - he was not real, he was merely an expression or avatar of the Matrix, and could appear as one or many, and the more he was, the more threatening. Generic name, multiple copies translates as a threat, or at least diminishes trust.

Names should be unique and informative, not generic, if their referents are to provide important services you want strangers to trust.

Okay, that's enough for now, time to go get some hot dogs and a little beach time.

All Spammed Up: Spammers Already Exploiting Michael Jackson’s Tragic Death

Fri, 26 Jun 2009 13:02:24 +0000

Just hours after Michael Jackson died yesterday, spam with subject lines claiming to have “exclusive information” on his death began flooding the net. The emails don’t contain any malicious links or attachments but seem to be an attempt to collect emails for a future attack. Researchers say anyone that replies to the spam will likely have their address harvested and that it wouldn’t be surprising to see future spams containing links to malicious payloads masquerading as exclusive video of Jackson’s last moments or autopsy photos.

News of the pop icon’s tragic death from what appears to be a sudden cardiac arrest caused an overwhelming spike in traffic that crashed Google, Wikipedia, AIM and Twitter for short periods and caused Facebook to slow to a crawl. Spammers and scammers are jumping at the chance to take advantage of all that traffic. Exploiting headlines and holidays is one of their favorite tricks. The last big headline they used was the Swine Flu outbreak, and before that President Obama’s inauguration.

Security experts are advising people to get their news only from reputable sources, and it goes without saying that you should never ever reply to a spam message. At best it will just bounce back due to a faked header, at worst it’ll just get you put on a list of people that respond to spam, meaning you’ll become a prime target for spammers.

John Graham-Cumming: Running the numbers on the BBC executives expenses

Fri, 26 Jun 2009 10:15:53 +0000

So, another lovely data set appeared, the expenses of senior BBC executives. And the papers went a little wild highlighting the spending that they don't like.

Me, I just punched the numbers into a little program and took a look at how well they fit Benford's Law. I wanted to see if there were any interesting anomalies to look at. And there are. No smoking guns, though. Just some fun on a Number 22 bus with some numbers.

First, here are the chi-squared values for the fit of first digits of the expenses for 2007/2008 by executive. The critical value (for p = 0.05) is 15.51, so most of the expenses do not fit Benford's Law. The fun is in finding out why.


Ashley Highfield,3.06874517880956
John Smith,4.61752636949634
Mark Byford,15.5817014457229
Jana Bennett,15.6178982350545
Zarin Patel,17.5034731417114
Jennifer Abramsky,20.8803339214804
Mark Thompson,22.4588511455346
Caroline Thomson,37.666988157616
Timothy Davie,143.433388695613
Stephen Kelly,178.662639451409

The best fit is Ashley Highfield's lovely curve:

We'll come back to Mr Highfield later, but let's go to the other end of the spectrum and look at the extremes that don't match the expected. The 'worst' offender is Stephen Kelly (he's not with the BBC anymore). Here's his curve.

Whoa. What happened with all those 8s? Delve into the data and you find lots of £8.00 claims for "Road/Bridge Tolls". My guess is that Mr Kelly passed through the London Congestion Charging Zone in his own car. That's enough to skew the data. And if you match up his £8.00 charges and his mileage claims it all makes sense.

Now to Timothy Davie and here's his curve:

So, he's like Stephen Kelly and sure enough there are lots of £8.00 charges for the same "Road/Bridge Tolls".

Next on ths list comes a different pattern created by Caroline Thomson. An excess of 1s:

She's got a ton of taxi trips in the £10 to £19 range. According to Transport for London you'd see those fares on a weekday when traveling around 4 miles in central London. Given the location of BBC Television Centre it's pretty easy to imagine the need for these trips. Also, she doesn't claim any mileage or congestion charge so she's not using her own car.

Next up is the Big Kahuna Mark Thompson. His curve shows an excess of numbers 6, 7 and 8.

Why is that? Well, if you look at his expense claims line by line (and if you do you're a total nerd) you'll see that Mr Thompson takes people out to lunch a lot and spends a lot of money on lunches under £100. You can imagine this being totally legitimate. He probably has to do that for his job, there could be a BBC guideline about how much to spend on lunch, or Mr Thompson could simply have a moral compass that says he shouldn't go totally wild on lunch costs.

He doesn't have lots of taxis or tolls, but then again there's a note in his expense report where he did take a taxi that says "Driver not available" so I'm guessing he has a chauffeur.

And so it goes on. You can carry on down the list and look for little anomalies, but there's nothing glaring.

So, how come Ashley Highfield has such a perfect curve? Well, he doesn't take a lot of cabs (so no excess of 1s), drives his own car (lots of little mileage claims) and doesn't seem to claim the congestion charge (no excess of 8s). Did he forget to claim the congestion charge, or does he drive an electric car?

He was, after all, the BBC's Director of the Future Media and Technology.

Enemieslist: A couple more: kudos, and mixed kudos/gripe

Thu, 25 Jun 2009 19:24:01 +0000

Another ISP with the right idea: floodcity.net. Clearly delineated both assignment type (in the right MST position) and technology:

64-186-88-109.dialup.dynamic.floodcity.net [64.186.88.109]

One which has sort of the right idea: musfiber.com

110.113.dynamic.musfiber.com [63.246.246.110]

Unfortunately, they've also done this to delineate their pools:

116.ippool#1.musfiber.com [63.246.240.116]

In the original RFC that defines the limitations on DNS labels, RFC 1035, names are limited to alphanumeric ASCII characters and a hyphen:

The labels must follow the rules for ARPANET host names. They must start with a letter, end with a letter or digit, and have as interior characters only letters, digits, and hyphen. There are also some restrictions on the length. Labels must be 63 characters or less.

The 63-character length limit has been eased, and some movement towards i18n has been made, with the introduction of punycode, RFC 3492, but even punycode limits itself to alphanumeric ASCII and the hyphen while encoding the other characters. So, as with underscores, octothorpes (aka "hash marks", AKA "sharps", AKA "pound sign", etc.) are non grata in DNS labels.

And yet, they show up occasionally, as in one of my favorite abuses of the DNS ever (blissfully now repurposed as a Windstream dynamic IP with sensible PTR naming):

spam.complaints.(888)292-3827.alltel.senior.support.ticket#2-940727661 [216.96.36.58]

On the bright side, they did give you a phone number to call. To report network abuse. Excelsior!

Enemieslist: A rare kudo: cgocable.net

Thu, 25 Jun 2009 18:57:55 +0000

I'd be remiss in my role as goad if I didn't occasionally reward those with the right ideas about PTR naming conventions, so let me just take a moment to say that the folks at COGECO Inc., a Canadian cable television and Internet service provider, have the right idea:

d141-145-153.home.cgocable.net [24.141.145.153] d38-5-18.commercial1.cgocable.net [72.38.5.18] d38-84-122.wifi.cgocable.net [72.38.84.122]

They've clearly delineated between their commercial, residential, and wireless services, and though they don't explicitly state whether each is statically or dynamically assigned, we still have enough to go on with respect to setting policies. If they want to be even more explicit than that in the future, then great!

Enemieslist: Today's DNS Spotlight: Eircom

Thu, 25 Jun 2009 15:54:13 +0000

Ever wonder exactly where some infected computer is, when you find it is hitting your inbounds with repetitive requests to send mail to nonexistent accounts, as in where in the world, down to ICBM coordinates? Yeah, me, too. Unfortunately, despite the best efforts of groups like the the Prefix Whois Project, who provide eerily precise longitude and latitude for any given IP, geolocation is still an infant science. Usually, the best that can be done is to provide the ICBM coordinates of the company providing the service, which while satisfying to an owner of fantasy desktop nuclear weapons, isn't quite as satisfying as taking out the actual infected computer while leaving all else around it standing. Oh, well. Maybe someday.

On the other hand, some networks do make an effort to name their systems so that they could be found by, say, firefighters. Take, for example, this wifi node in a McDonald's in Dublin, Ireland:

83.70.120.247-dynamic.wlan-ce1.mcdonalds-50-lower-oconnell-st.cust.eircom.net [83.70.120.247]

Now, I've never been to Lower O'Connell Street, nor Dublin, nor Ireland for that matter (though my sainted Grandmother Betty was swept away as a war bride during the second World War, from her ancient homeland in Fintona, County Tyrone). And the pwhois coordinates for this IP are a good seven minutes drive on the south side of the Liffey from Lower O'Connell Street. But still. One imagines a well-launched predator trained on the wifi node, or maybe the more imaginative can picture a Terminator preparing to tap a customer replete with laptop and large fries, and a more satisfying ending (perhaps involving a Taser).

What's sad, as anyone who's been reading will instantly recognize, is that the "dynamic" keyword is a far cry from the Most Significant Token, and is not itself actually tokenizable by the weak, dot-delimited (and limited) MTAs we're blessed with, so in order to recognize this is a dynamically assigned IP you must use a regular expression. You may be able to use a substring based on "wlan-ce1", but still there's this business of a complete street address and business name to contend with. Do we need to keep track of the wifi node at Govinda's, the vegetarian place around the corner, too?

I don't know if Govinda's is an Eircom customer, but the Citywest Hotel is:

213.94.167.154-dynamic.wlan-ce1.citywest1-hotel-dublin.cust.eircom.net [213.94.167.154]

Now, we could, I suppose, just use "cust.eircom.net", but that is less than satifying (and they also use "customer.eircom.net", or used to). And it highlights another problem - just because an ISP or telco assigns an IP statically to a customer doesn't mean that customer can't then go on to re-allocate it dynamically to Big Mac eating, laptop-wielding, customers of theirs.

(A brief scan of the PTRs in 83.70.120.0/24 informs us of the depressing reality: there are several McDonald's on O'Connell Street alone... In fact, the lower /25 of that block seems dedicated to the McDonald's restaurants of Ireland.) On the other hand, they all seem to have wireless, so, make of that what you will.

So where does that leave us? We know the IP is dynamic; we can use a regular expression to capture many similar instances, provided that Eircom sticks with its naming convention for building out wireless LANs; we can even get within a few miles of the actual location if we wanted to launch an imaginary missile. But it would be so much easier if Eircom just used "dyn.eircom.net", or even "dyn-wlan.cust.eircom.net"; they already know it's going to be handed out dynamically via wifi. Why not just say it and group all similar IPs under the same top-level token?

Box Of Meat: BBC NEWS: Habitat sorry for Iran Tweeting

Thu, 25 Jun 2009 15:23:52 +0000

BBC NEWS: Habitat sorry for Iran Tweeting:

“Furniture store Habitat has apologised for causing offence after accusations it exploited unrest in Iran to drive online Twitter users to its products.

…When asked whether an outside firm had been responsible for the strategy their spokesman declined to give details.”

By refusing to out the marketing firm that made this mistake, Habitat is basically guaranteeing that another company will get caught up in the same mess again in the future.

Box Of Meat: Cisco: DomainKeys Identified Mail (DKIM) Grows Significantly

Thu, 25 Jun 2009 14:01:33 +0000

Cisco: DomainKeys Identified Mail (DKIM) Grows Significantly: some interesting graphs

All Spammed Up: UK Furniture Company Apologizes For Exploiting Iran Conflict in Twitter Spam

Thu, 25 Jun 2009 12:25:52 +0000

British furniture retailer Habitat has apologized for exploiting the Iran conflict in an attempt to promote its Twitter feed. The company came under fire after it began using keywords related to the current conflict in its tweets, which otherwise had nothing to do with the subject. This is referred to as hashtag spam and is widely frowned upon by Twitter users. The company also used other high trending keywords such as #Apple and #iPhone.

Sky News Online has reported a Habitat spokesman as saying: “This was a mistake and it is important to us that we always listen, take on board observations and welcome constructive criticism. We will do our utmost to ensure any mistakes are never repeated.”

The company has not issued an apology on Twitter but did quietly delete all the spam tweets it posted. It’s not clear why they felt hashtag spamming was okay to do, although they told a blog that it was done without their knowledge. That sounds a little hard to believe but it wouldn’t be the first time a rouge employee was blamed for a blunder that became a PR nightmare.

The moral of the story? Twitter can be a valuable tool to help you reach out to customers and potential customers, but tread carefully and follow the rules. Spam is no more acceptable there than it is anywhere else.

John Graham-Cumming: Britannica.com makes me want to weep

Thu, 25 Jun 2009 12:15:29 +0000

I got a marketing mail from Britannica.com trying to entice me back after I canceled my subscription. So, I figured I'd just go take a quick look at a random Britannica entry and remind myself of what I was missing. Nightmare.

On the Britannica.com home they were mentioning that their article about the US Voyager program was featured and I could see it for free. So I clicked.

This featured article contains 503 words that give the briefest of introductions to Voyager. The related articles are all about the planets that Voyager passed, and there's a connection to a general article about space exploration. There's absolutely no drill down to explore Voyager in any depth.

Of course, I whizzed over to Wikipedia and looked up the same subject. The main article contains 2,009 words and links to in-depth articles about Voyager 1 and Voyager 2. And there are links to interesting articles about their voyages, their power systems, the Voyager Golden Record and more.

And Wikipedia links you straight to the definitive source for Voyager information: NASA's Voyager Program page. Britannica doesn't link; they choose to link to a small collection of images of the Voyager craft from NASA's web site.

So, basically Britannica.com's article is close to useless because it's a dead-end and a short dead-end at that. In contrast, Wikipedia's article is rich, links to even more information and lets me get to source material.

And if that's not enough Britannica.com's page is infested with distracting ads. The worst of these are the weird keyword-linked ads buried right inside the article itself.

It looks like you might be able to click on, say, solar system in the article to drill down. Far from it! Hover over solar system and you get the following irrelevant, useless, pop-up ad.

Pure genius, Britannica.com. Pure, pure genius.

Now, Britannica.com's article does contain some drill down, but some of it is useless. For example, the Voyagers each contain a phonograph record with a recording of sounds from Earth (language, music, etc.). On the Britannica.com page the words phonograph record are a link. Click through and they will tell you what a phonograph record is, not about the ones on board the Voyagers. Thanks, I'm old enough to know what a phonograph record is.

So, Britannica.com, now you know why I donate money to Wikipedia, and don't buy your service.

John Graham-Cumming: Michael Faraday criticizes 'security theatre' from beyond the grave

Thu, 25 Jun 2009 09:55:13 +0000

I was reading David Knight's book Humphry Davy and at one point he describes the arrival of Davy and Faraday in France in 1813:

On arrival, Faraday reported, they were searched, an unusual experience for a true-born Englishman: 'he then felt in my pockets, my breast, my clothes, and lastly, desired to look into my shoes; after which I was permitted to pass', and could hardly help 'laughing at the ridiculous nature of their precautions'.

Lucky he doesn't have to fly anywhere.

Justin Mason: Still using perl 5.6.x?

Thu, 25 Jun 2009 08:50:02 +0000

For the upcoming release of Apache SpamAssassin, we’re considering dropping support for perl 5.6.x interpreters. Perl 5.6.0 is 9 years old, and the most recent maintainance release, 5.6.2, dates back to November 2003. The current 5.x release branch is 5.10, so we’re still sticking with a “support the release branch before the current one” policy this way.

If you’re still using one of the 5.6.x versions, or know of a (relatively recent) distro that does, please reply to highlight this….

Terry Zink: Is Twitter spam possible?

Thu, 25 Jun 2009 05:16:02 +0000

With the explosion in popularity of Twitter (of which I am not a twitterer or even a subscriber), I've wondered to myself whether there is such a thing as twitter spam.

Now, spam in the email sense is when spammers flood your inbox with unwanted email. But with Twitter, if you're subscribing to someone's feed, then how can you be spammed? You could just stop subscribing them if they were really getting annoying but really, you're opting in and you know who's sending you "mail." It's kind of like getting RSS spam... which is counterintuitive.

I did a quick Bing search and found out that there is such a thing as Twitter spam known as "Follow spam". From Twitter's blog:

What is "Follow Spam?"

Follow spam is the act of following mass numbers of people, not because you're actually interested in their tweets, but simply to gain attention, get views of your profile (and possibly clicks on URLs therein), or (ideally) to get followed back. Many people who are seeking to get attention in this way have even created programs to do the following on their behalf, which enable them to follow thousands of people at the blink of any eye.

As you can imagine, this is a problem. In extreme cases, these automated accounts have followed so many people they've threatened the performance of the entire system. In less-extreme cases, they simply annoy thousands of legitimate users who get an email about this new follower only to find out their interest may not be entirely...sincere. On rare occasions we may see a person who is mass following and actually cares about every tweet—there is an opportunity for us to learn more about this use case and work to provide a better experience.

I don't fully understand why someone would choose to engage in Twitter spam but the idea seems to be that if you follow a lot of people's Tweets, the followees will click on your profile. If you were a spammer, you could a link to your product in your profile in hopes of getting the Tweeter to follow it and get to your site. It's a way of avoiding a spam filter since the spammer is already in the network and presumably, there is a level of trust. After all, if your a Tweeter, it's kind of flattering to have a lot of people follow your tweets.

But for the Tweeter, having a lot of spammers follow you becomes really annoying. You want real people to follow you, not spammers hyping up your statistics. You can't go through your followers profiles because all you're doing is sifting through a lot of chaff. Twitter also cannot build accurate statistics on user profiles in order to one day monetize their size.

Ultimately, the problem of Twitter abuse will come back to the same problem faced by the webmail providers - spammers are breaking CAPTCHAs and using them to send out spam. The spammers are doing the same thing here an irritating everyone with their abusive behavior. I suspect that there will be a similar convergence in anti-CAPTCHA-breakage techniques that there was for spam including IP reputation and behavioral analysis (content filtering).

Enemieslist: new pats posted - 20090624-01 (maintenance pats release)

Wed, 24 Jun 2009 23:02:02 +0000

39299 patterns, 11333 right anchor strings, 170516 test IPs.

More from a CBL list.txt I recently resolved down to PTRs. This release
matches 99.995% of the PTRs in that CBL zone.

Was asked to start tracking couplets (pattern class and tech, taken
together as a sort of meta-identifier); there are no new couplets in
this release.

Download them here:

sendmail:
http://enemieslist.com/downloads/sendmail_access_db
http://enemieslist.com/downloads/rightanchors

postfix:
http://enemieslist.com/downloads/postfix_regexp_table
http://enemieslist.com/downloads/postfix_regexp_table-20090624-01

exim:
http://enemieslist.com/downloads/exim_hosts
http://enemieslist.com/downloads/exim_hosts-20090624-01

Ed Falk: Meanwhile, Ralsky apparently gets to keep most of the money

Wed, 24 Jun 2009 23:28:07 +0000

Yesterday, I wrote that spammer Alan Ralsky had pled guilty to a number of charges, and was facing roughly 3½ years in prison. The latest word comes from an FBI press release which indicates that Ralsky is also facing a $1 million fine.

However, Ralsky is said to have made $3 million on his various scams.

Hmm, let's see ... $3 million minus $1 million — carry the eleven — is wow, a whole lot of money. Not bad wages for 3½ years.

I'll leave it to my readers to draw their own conclusions about the U.S. justice system. I've already drawn mine.

Justin Mason: Links for 2009-06-24

Wed, 24 Jun 2009 22:05:03 +0000

Brian Krebs on the Ralsky guilty verdict : good quote from Richard Cox of Spamhaus: “This has been a long time coming. Ralsky has been identified as one of the key drivers of [..] development in the spam world [...] among the first to commission mass-mailing Trojans to help develop spam botnets.”‘
(tags: alan-ralsky stock-spam busts prosecutions guilty spam law spamhaus botnets)
Facebook stolen-account scam : a mate had his FB credentials stolen and the account used to attempt to scam his social group. Sample chat: ’so where should I send the money?’ ‘you can have it sent to my name and my present location [...] Do you know any western union outlet nearest to you?’
(tags: western-union scams facebook security phishing 419 social-networking)

Enemieslist: Today's DNS Superstars: Entel Chile

Wed, 24 Jun 2009 21:26:32 +0000

Take a look at the PTRs in 200.72.157.0/24 for a stellar example of why there should be a basic intelligence requirement for anyone touching the IN-ADDR.ARPA zones for any given ISP.

200.72.157.1: pcsistema1.eiser.local 200.72.157.4: pcproyectista1.eiser.local 200.72.157.5: pcsistema3.eiser.local 200.72.157.6: christian.christian 200.72.157.7: pcsecretaria.eiser.local 200.72.157.8: pcdibujante2.eiser.local 200.72.157.9: pcproyectista1.eiser.local 200.72.157.11: pcsecretaria.eiser.local 200.72.157.12: pcsistema1.eiser.local 200.72.157.13: regiones-serena.previred.lan 200.72.157.14: pcproyectista2.eiser.local 200.72.157.15: pcsistema1.eiser.local [...] 200.72.157.247: pcdibujante2.eiser.local 200.72.157.248: pcsistema4.eiser.local 200.72.157.249: servidordsn.datasoftnet.local 200.72.157.250: pcdibujante1.eiser.local 200.72.157.251: pcproyectista1.eiser.local 200.72.157.252: regiones-serena.previred.lan 200.72.157.253: pcsistema4.eiser.local 200.72.157.254: pcdibujante2.eiser.local

I know, .local is actually a valid DNS suffix, in the context of zero-configuration networking discovery protocols, such as Bonjour or Microsoft Windows Active Directory, if the host is never to be visible from the public Internet. The presence of a few dozen of these very hosts in the CBL zones, however, suggests that something is wrong here, and they are actually capable of reaching the Internet directly. Nice work, guys.

I really should work up a badge for these sorts of things, maybe a bear trap containing a keyboard with hands caught in it.