wiredfool

About a month ago, I added centralized spam scoring to my mailserver using the latest (2.55?) spamassassin. Like most people, I’m worried about missing real mail if I set the drop threshold too low. Currently I’m killing 5-10% of the most blatant spam (and 100% of the email worms, the original reason for adding filtering serverside). Now that I have a month of logs, I can see what the incoming email looks like.

This graph is (sort of) a histogram of the spamassassin scores for all incoming mail to my server that didn’t get killed in the initial virus/worm scan. The red line is all mail, the green line is traffic to addresses that get nothing but spam. It covers about 11 thousand messages total, 2 thousand are to the spam-only addresses. Unfortunatly for the analysis and my sanity, I don’t have an easy way to find the spam scores of known good email. Maybe I can correlate message ids in my mailbox with records in the logs.

The known spam certainly has what looks like a similar distribution to the tail of the main curve, leading me to suspect that these addresses attract a reasonably representative sample of the more blatant spams. About 10% of these emails fall under the traditional ‘5’ threshold for spamassassin.

There are a few peaks in the probably good region, I’m betting that at least one of these corresponds with a high volume mailing list that I’m on.

So what does it all mean? Spamassassin isn’t going to cut it without the bayesian type filters that are all the rage now. But for the bayesian thing to work, you need training, feedback and individual preferences, and that’s just not going to work at this layer of the stack. It’s time for lateral thinking.

* It’s a histogram graphed as lines because boxes just looked like too much chartjunk. It really shouldn’t be using solid lines, because it’s not a continuous function. But the area mass actually is pretty close to what I’m looking for. With some normalization, it could look like a probablity distribution function. (or a cdf, which is as useful for finding cutoffs)

So much for the wireless revolution. I’ve returned the pda and cancelled GPRS, since as far as I can tell, I’m just not going to get the experience that I was looking for.

I spent a couple of hours on the phone with t-mobile support establishing that the abysmal performance of their proxy server wasn’t just a temporary cell crowding thing, a ppp settings thing, or anything else that made sense. We finally got it working at somewhere between a 9600 and 14.4 modem, but with a 30 second lag for the first page to hit the proxy. Turning off the proxy by going through a different AP helped with the latency and speed a bit. Until they went through billing to make the change permanent and managed to switch me back to using the proxy again.

The final verdict is that it’s not supposed to be this slow, and that maybe it’s my phone. I’m going to leave it there, I’ve spent too much effort to justify the marginally better-than-a-modem service that GPRS provides. I suspect that I’d have a different impression of GPRS if I were using a dedicated non-phone device like a danger or blackberry. Email and chat are much more tolerant of the low speed high latency connection than anything interactive like ssh. (SSH and google being the two things that I really want in a phone/pda combo.)

Maybe next year.

If you haven’t heard yet, Verisign made a dns change yesterday that replaced the ‘domain not found’ error with a record that pointed to one of their servers. The net.world is up in arms, ISC has issued a patch for BIND (and the announcement made it into the dead tree Seattle PI today), and no one is generally happy about it.

I didn’t realize how bad it could be until I saw this post from Steve Bellovin on NANOG:

It’s bad enough now; it could be even worse. They could respond on
port 443, too, with a legitimate-seeming certificate — they’re
*Verisign*, the leading certficate authority.

In the security world, we call this a man- (or monkey-)in-the-middle
attack, for which the standard defense is crypto. But that doesn’t
work well when your trusted third party is part of the threat model…

I’ve never really liked that there was one central authority for public key certificates, and I really don’t like that they are in control of other central parts of the infrastructure, and even worse, that they have been taken over by people looking to make a fast buck at the expense of the net.

So far, I’d say that the next generation wireless web is just not there yet.

The speed is glacial, connections are not exactly always on, and there’s been more fiddling than should be required.

At this point, I can:

• Establish a gprs connection if I have just rebooted both the phone and the pda.
  I can make multiple connections from the computer, just not the pda.

• Keep the connection going for hours, blocking incoming calls
• Websurf to anything other than port 80
• Check email

What I can’t do is go to anything on a web port, because t-mobile redirects all access through their proxy. This proxy is supposed to speed things up, but I’ve yet to have it return a page in less than 5 minutes. Pages requested on non-standard ports (such as Radio Userland’s 5335) work in seconds. Slow, and modem like, but they come up. I think I could fix this by installing squid on one of my machines at home, but I’m pretty sure T-mobile does not expect their customers to be setting up their own proxy servers so that they can get some surfing done. And if they do?

And the speed. … I’m not actually how this is any better than using the single channel 9600 baud modem in the cell handset.

I expected to do some work to get this to work. I expected that there would be ‘non-standard platform’ issues. But so far the payoff for the work has been pretty minimal.

Zaurus -> bluetooth -> GPRS -> world.

Wireless Fool!

Disclaimer, this post was edited later on the laptop.

Update some days later – that was also the last post.

Now for my computing pleasure (or frustration) — a sharp zaurus, bluetooth compact flash card, bluetooth phone, and bluetooth dondle for the laptop. In theory, this is enough equipment and service to get online from anywhere and have a full IP connection anywhere I can get cell service.

So far in practice, there are snags.

The biggest snag is that there are two AmbiCom BT2000 compact flash bluetooth cards, the BT2000C and the BT2000E. The C version is apparently not supported under linux (the Zaurus) while the E version is. (notes here and here and a comment) Amazon’s product pages don’t mention the version, but from what I can gather online from people who’ve tried this, the E version is yellow and black, the C is blue. Yep, Amazon’s picture is yellow, the one that shipped is blue.

In keeping with my tradition of reformatting any computing device I get within the first three hours, I installed the updated rom on the advise of the zaurus community. It’s a little comfusing, as the docs menton a “romimage” file that is actually named Ospack. After deciding to risk turning a 1 hour old device into a brick with a not-quite-correct rom image, it did install correctly, but rebooted before I could pull the compact flash card. This helpfully spewed fsck errors and dropped me into file system repair. Pulling out the card fixes that one, but I’m not sure that a palmtop really should have the whole fsck thing.

And in other Zaurus news, don’t pull the battery and the AC to reset after a crash. (well, don’t crash it by just trying all the bluetooth modules either). You lose everything but the roms, which means things like terminal and the bluetooth drivers and the config files. Yay.

The phone (Ericsson r520m) and computer seem to work reasonably well together, after 45 minutes with techsupport and hold music at t-mobile. It’s not quick – bordering on 14.4 modem speeds – but it does work through either irda or bluetooth in either dialup or GPRS modes. Tech support needed to walk me through getting connected through wap so that my handset was recognized by the system. It’s relatively straightforward given the tmobile configurator and a the ip address of the wap system.

UPS is supposed to be bringing toys today. They’re not here yet. I hear every big truck that goes down the street, and none of them have been big and brown. But I still have to look every time.

Our sunflowers are starting to come out.

This one is doing a chinup over the tree branch.
(yes, I have flowers growing up into the trees)

And this one is just starting…

I wonder if it’s possible to use a bluetooth cellphone as a headset for a bluetooth enabled laptop connected to iChat A/V (possibly at a cafe using wifi?). The hardware is there, it’s probably just a matter of settings/software. Or a bluetooth phone with a compact flash/secure digital wifi card to dispense with the laptop? There are some pocketpc/handspring things that are close to that hardware configuration.

I’ve finally given in and broken the end-to-end assumption in mail transmission by installing an antivirus/spam filter on my domain’s mail server that drops all (windows) executable attachments and any spam over 15 on the spam-o-meter. (3x the default threshold) I’m writing some instructions on how to do this on debian woody, as it requires pulling some packages from unstable & backports. After 250 viruses and 300 spams dropped in a week and a half, I’m pretty sure that I haven’t killed anything important.

I ordered a new video card for an ancient sawtooth 400mhz G4. It’s amazing how much snappier the NVidia Geforce4 feels than the stock ATI Rage 128. I’m pretty sure that most of the difference comes from Quartz Extreme usage and not that it has 4x the vram than the old card. But it makes Eclipse usable, which is not really something that I can accuse my tibook of doing. The DVI/VGA adapter did have one interesting feature, apparently if I remove the UPC sticker from the box, some warranty is voided.

A random house plant, caught in the light of a setting sun.