wiredfool

The Problem

On Osx, Frontier runs as a user process, not as root. Therefore, it can’t bind to ports below 1024. We have been using port forwarding to forward low numbered ports to high numbered ports. This is a manually maintained script that resides in /Library/StartupItems/Multihoming/.

There are 2 major drawbacks:

• It is manually maintained, and I’m tired of doing that for complicated systems.
• There is an issue with loopback tcp where either the operating system or applications that don’t know about port forwarding assume that [publicip]:port is equivalent to [loopback]:port. If frontier can parse the forwarding data, then it can do the right thing. We are seeing this with xml-rpc requests from instance to instance.

Proposal

Automated Maintenance

There should be a frontier tool that can make the Multihoming directory, write all associated scripts, and add a file with its information into a subdirectory of the Multihoming startup item. Sudo should be used as necessary to write the config files, but the datafiles should be writable and readable by user processes. A sudo execution shall not be necessary for frontier to read this data. The port forwarding script will have to be run as root, so it should be triggered by a menu item or run from the command line.

The datafiles should be text files that are named for the ip address being forwarded and one line per port forward request, source port [space] destination port. Lines should be written in unix native format. Comment lines are delimited by #. The multihoming script should ensure that all ip addresses referenced are aliased to en0. (Aside, should there be seperate configuration directories for en0,1,2?)

Configuration data should be stored in config.root.

Loopback Bug Fix

When Frontier starts up, and every so often thereafter, it should read the datafiles and parse them into an internal structure. When making a tcp connection, Frontier should examine this structure and determine if the connection end point should be rewritten to a local high port. This change will probably make sense in tcp.httpClient, but could also be added to tcp.openStream and tcp.sendMail. This should handle communication between one frontier process and other processes on the same machine.

There should also be recognition of the apparent ip address of the http inetd listener so that communication within frontier correctly intreprets if the request is loopback or not. I propose the following:

1. request port == apparentPort and request ip == apparentIp -> process internal loopback
2. request port == port and request ip == loopback -> process internal loopback
3. request port == apparentPort and request ip == apparentIp -> process internal loopback
4. request ip, request port are forwarded -> machine loopback to target port.

Must… Finish… Taxes….

Every year it’s something else, every year I think that next year’s taxes are going to be simpler. And every year, I’m wrong, at least for the last 4 years running.

First it’s a house. Then it’s a bit of consulting on the side. A jump to a different job with different retirement accounts. Then inconsistent accounting at the outsourced HR company. The company nearly dies then crawls back to life.

At least I don’t owe. I’m figuring that each hour of tax preparation misery should be worth at least $100 of refund. Of course, they’re paying me with my own money, so they should be able to afford it.

For one brief shining morning, my laptop was the sole machine on the end of a brand new t1 that needed a little load testing.

That contented sound you hear is the sound of high quality of service bandwidth sucking. I know it’s not capital B broadband, but it’s certainly a change from cable and congested dsl.

Now if they’d just pull 10 or 100 Mbit fiber over here. I don’t ask for much. I just want it all, and I’d like it now please.

***T1s for <strikethru>dummies</strikethru> software people

Vocabulary…

• Demarc – that little box on the wall that the phone company put in in the wrong room that takes 2 wires and turns them into a rj45/48 connection.
• Extended Demarc – The bailing wire and punchdown block contraption that you assemble that is essentially a 150 foot straight through cable. (pins 1,2,4,5 go straight through, otherwise known as the the unused pairs from an ethernet cable) This would not have been necessary if the phone company had put things in the right location in the first place.
• CSU/DSU – What you plug the extended demarc cable into. Looks like an ethernet port, but it’s not. In this case it’s a card in the back of the router.
• Router – The thing that appears to nearly work, but doesn’t until the guy on the other end flips a switch. Probably a cisco with a 68000 class processor running IOS.

Despite this tale of apparent confusion, it’s actually easier to get one of these installed than to get DSL from Quest. You sign a big contract, confirm for 7 people that you are in fact the contact, deal with phone company confusion after the fact, get badgered by the engineer, deal with your own lack of knowlege, get the engineer on the other end to do his magic, then it works.

Whereas DSL takes many calls, some rescheduling, badgering, and then when it doesn’t work, you have to call a customer (dis)service rep. When you’ve finally returned from the 7th circle of voicemenu hell, you might have a net connection. If you’re lucky. And sacrificed a squash to repent for your sins.

*** Oops.

Gems and static pics were down for a while while apache was listening on the dsl line, but dns was pointing to the t1. Guess that’s why you don’t test on client sites…

After living with a digital camera (Cannon Digital Elph, S-100) for a year and a half, I have a short list of what I want in my next one.

• Better low light performance. ASA 100 (f2.8-4.0) barely cuts it outside in Seattle, and I have artistic objections to using the flash. I’d be willing to trade resolution for sensitivity for real low light shots. To do this, I’d need much better exposure control.
• Better usable resolution. 2.1 megapixel is good enough, if that’s the real resolution. If you look at the results, I don’t quite get that. The Foveon sensor might help that, as would more raw resolution.
• More saturation. It seems that this camera always needs a bit of a boost to the saturation to come close to the picture that I remember.
• Longer lens. I’d like 300mm, f 2.8, and have it fit in my pocket. I know I’m dreaming there.

Oh yeah, and I want it all in the same digital elph form factor so that I always have it with me.

The weather has changed. It was a glorious beginning of spring: sun, flowers, fresh air. But last night, change blew in and now it’s just grey spring rain.

Basic biscotti, hacked from a recipie on allrecipies.com. Makes about 3 days worth of biscotti for 2 people.Oven @ 375, mine runs a little hot I think.

Mix: 1/3 cup oil, 3 eggs, 1+ cup sugar, 4 teaspoons assorted extracts. (vanilla/almond/etc)

Add: 3 cups flour, 1 tbsp baking powder. Add couple handfuls chocolate chips. Mix till uniform.

Parchment paper a cookie sheet, roll and flatten to 1/2 inch thick. You’re going to cut across the width, so if you like small biscotti, you may want two sheets, or multiple smaller blobs of cookie.

Cook 25 minutes, till golden. Take out, cool enough that you can cut into 3/4″ thick slices. Arrange cut side up, cook 5 more minutes on each side. The biscotti should be crispy.

I’ve released my Patch Tool. It’s the first release of a developer oriented tool that provides patch and diff services for Usertalk developers.

If you’re on OSX, it will do all the work on your machine. If you’re not, it will contact a web service running on OSX to do it for you.

Version: 0.16, 9/5/2002

Download: http://radio.weblogs.com/0001179/gems/patch.root
To Install: Put in your Tools folder in Radio Userland or Frontier.
License: BSD. Copyright Eric Soroos, 2002. Released with the permission of SocialEcology Inc.

***What it does

This is a developer tool that provides patch, diff, and cvs integration services for Usertalk scripts in Frontier and Radio Userland. Its integration level is currently roughly that of app glue. There are plans to use this core to provide comprehensive source management.

Starting with release 0.16, there is code to dump the contents of a (nested) database table to disk in text format and to read it back in again. The files are written in a way that allows interoperation with conventional source code management systems (e.g. CVS). This could also be suitable for backup of ueser preference settings. Scripts and outlines are written in opml format, binaries and menubars are written using base64 encoding, everything else is written as plain text.

If you are running on Mac OSX, it will do the diffs and patches on your local machine. If you are not, it will contact a webservice running on differenceEngine.wiredfool.com which will perform the differences for you.

***How to Use

There are two entry points for the difference code:

• diffString = patchSuite.getDiff(@original, @modified)

  GetDiff will return a context diff of the xml representation of the script. You will probably want to edit the diffString to not include spurious differences in the header properties.

• errString = patchSuite.applyDiff(@original, @patch, @patchedScript)

  ApplyDiff takes the context diff created in step 1 and applies it to the original script, then places the result in a new script and compiles it. As far as I know, the only formatting that is lost is the outline expansion state. ErrString is the result from the command line call to patch.

There are also two entry points for the file system code:

• patchSuite.dumpToFileSystem(@table)

  Dump to file system will prompt for a folder to save the contents of the table. It will write out the contents of the table to the file system, With the exception of items with a tab in the address. This excludes the compiled versions of xml items.

• patchSuite.readFromFileSystem(@destination, flIgnoreCVSLint=true)

  ReadFromFileSystem takes the items in the file system and assembles them into the table at the destination address. The one optional parameter ignores all directories named CVS, which are status repositories for the CVS system and not applicable to Frontier.

***Tweakable Bits

You can change the rpc server at patchData.prefs.rpcServer if you don’t want to use mine.

***Known Issues 9/5/2002

• (Dump) There should probably be a flag to ignore the tab characters in addresses.
• (Dump/read) Has been tested well on script, outline, string, boolean, and number types. It chas been tested less well on lists, binaries, and menus (but should still work). It loses formatting on wptexts. It should work on most other types, but the exotic ones have not really been tested. There are probably going to be issues with items in lists where the item length is > 256 characters and anywhere else that [itemTypeCast](string(item)) isn’t an identity relation. Restoring addresses may require that I strip the root portion off first.
• (Dump/Restore) Fails when trying to dump the running script. Probably will die if it overwrites the running script on a restore. Workaround: copy the script to system.temp and run from there.

I got slashdotted today. Well, not exactly slashdotted, and not my site, but a server I run. But none the less, one of my servers was hammered due to an external media event. And since there’s no mushroom cloud over the hill, you can tell that everything survived.

Loads on that machine were running 3-4 times normal peak loads, effectively maxing out the machine for 30 minutes at a time. Sometimes it was churning smoothly but a little sluggishly, and sometimes it was dragging with far too many queued requests for good quality of service. Looking through the logs, the difference in load level between these two modes was minimal. In the most extreme case took less than a minute to switch from one mode to the other. Or rather, it looks like it took one 15 second page load to slow enough other requests that the requests bogged down in a chain reaction accident for the next 30 minutes.

One request out of thousands. One driver hits the brakes. One pebble in the stream. One puff of air. All enough to take a supercritical flow and kick it from laminar orderly flow to turbulent chaotic flow. Like the fluids gurus, we can find the little chaos generators and painstakingly remove them from the system. But Chaos is a difficult mistress. She shows up in simple systems, and no matter how hard we work, we can only push back the boundaries of the chaotic zone.

Not that I’m going to call my web apps simple systems. Or that I’m going to give up on removing turbulators. A cached value here, a super-plasticizer there, and a little bit of Moore’s law over in the corner. There’s progress to be made shoveling back the chaos.