wiredfool

Sometimes, memory is slower than a file.

In Python, there are ‘file like objects’ that all respond to the same sorts of methods, open, read, write, close, that sort of thing. Some of them are files on disk, and some are memory buffers that are implement those methods. One would generally think that memory access is going to be faster than file access, but in the case of cStringIO and csv, it doesn’t appear to be the case.

I had a 4000 line csv file, about 700k worth, and another one that was 2x the size. Parsing that file took 10 seconds, the 8000 line file took 40 seconds. (amd64/ubuntu linux/python 2.4). Holy O(n^2) Batman! Since the CSV is just reading lines off a file (which implicitly advances the file pointer), I’m guessing that the cStringIO implementation of a file pointer is a numerical offset followed by traversing the string till it gets to the right spot. It’s not an actual pointer to a spot in memory.

So, I ripped out this code and used a temporary file (which on this machine, is going to be memory backed until it gets swapped out), and the times went to 2 and 4 seconds respectively. Almost certainly here, the file pointer is an actual pointer to a position in memory. Each movement is only a traversal of the size of the line, not the size of the file.

Big counterintuitive win here, by ignoring the buffering that Python provides and using the one that the OS provides.

OMG WordPress 2.2. Upgraded to it a few evenings ago, and just tried to post for the first time. I’ve just lost 5 minutes of my life trying to get the last post to show multiple paragraphs instead of just mashing into one while using Safari to edit. Sometimes I miss <a href=’http://www.userland.com/manila’>Manila</a>. I got real tired of hosting it and keeping it running, as it was forked from Manila of 5 years ago. But dammit, it tended to actually do what I meant it to do most of the time.

Edit: That actually looked coherent in the visual editor. I’m going to leave it as a protest. WTF kind of weblogging software doesn’t accept typed in links?

Oh, and furthermore, it’s painfully slow getting to the admin pages in Safari if you’re not using the Webkit Nightlies. Painful. in. my. default. browser.

Dear Apple,

Apparently you’re considering dropping the mini from your lineup.

In the immortal words of some space hero: Noooooooooooooooooooooooooooo!!!!

Seriously. These are probably my favorite desktop Macs ever. I have two PPC minis, and I’ll get Intel ones when I get too annoyed at how much faster/shinier/better my MacBook is. They are absolutely the right size and quietness for my office.

But if you do drop the mini, please consider shipping a Mac Nano in its place. Or a shuffle. I’d totally dig a Mac Shuffle with Wifi and Bluetooth, and only USB, Power and Monitor connectors.

Sincerely,

wiredfool

I was just browsing along, when this low budget ad caught my attention. It’s selling patches, or watermarks, or something.

An image at this resolution from istockphoto is $1, shutterstock takes a $199 subscription for the ability to download 25/day for a month. (And it’s on the first page of search results from shutterstock).

Perhaps the diet patch business isn’t as good as my email box would have you believe.

I picked up a Pantone huey colorimeter this week so that i run a little more of a chance of getting a reasonable match between the colors that I see and what comes out of a printer. I can’t say that I’m 100% convinced that I’m getting good results yet, at least on the MacBook’s screen. The MacBook vs. Huey is not exactly an unknown issue on the net, though this seems to be a more subtle version than the heavy green casts that people were getting.

First off, the colors are different. Cooler and less contrast are the first things that I notice. if I take a break and come back to to the screen, the colors do look reasonable, so that’s not evidence of miscalibration. I certainly wouldn’t be surprised if it turned out that the native MacBook colors are far punchier with high contrast and saturation and muddy shadows. That’s great for videos and selling machines in the store.

What does worry me is that the B&W images that I’m displaying (in Lightroom, a color aware application) aren’t coming out in pure neutral. There’s distinct coloration, a sort of off color yellowy green in the mid-tones. That shouldn’t be there in a properly calibrated screen. I’m going to venture a guess that the response curves for the individual RGB channels aren’t the same. The calibration software does the bulk of the profiling with neutral tones instead of a pure singe color. I’m not sure why, unless it’s a speed or product differentiation issue. That compromise would wind up making the overall luminosity curves correct at the expense of the color balance. Thankfully, it’s something that might be fixable in software.

The calibration seems better on my Dell 20″ desktop LCD, so perhaps it’s a quirk of the MacBook screen. I’ll have to try comparing to good prints to see if it can nail skintones on the external monitor. If it can, then it’s worth having.

When using cURL to upload raw csv over a HTTP post connection, the –data-binary option prevents munging the line endings out of existence. For some reason, even though files provided to the –data option are supposed to already be in their escaped form, it does another pass of something and returns line endings.

I’m talking on the phone and I hear this buzzing like the biggest bumblebee you’ve ever heard, and them some. But I know it’s not a bee, and pretty soon, I’m going to be fishing a hummingbird out of my skylight. He’s pretty unhappy about this glass ceiling thing.

It’s not hard to catch them, once they’ve settled down a bit and I’ve gotten up on the desk. A little matter of grabbing it in cupped hands and walking it outside. Then watching as the little guy shoots up 45′ into a tree where he started chittering at me.