wiredfool

I’ve had my own domain for nearly 10 years now. Somewhere early in it’s life, I did some things with email and spam control that were a reasonable idea then, but have now turned into something of a non-standard nightmare. Because of some early choices, it’s now nearly impossible to host my domain on shared hosting (i.e., someone else’s admin work) without completely changing all of the email addresses that I’ve given out over the years.

Along the way, I’ve had throwaway addresses for posting on usenet. (that accidentally leaked my real email out one time when I wasn’t careful). The throwaway emails turned out harder to throw away than I’d hope, due to the software that I was using.

So, enter new software, and I implemented a recipient delimiter scheme. Base-extension@domain.com. Only, what I should have done, but didn’t, was use the default delimiter of my MTA of choice (postfix) instead of a non-standard one. Even then I didn’t have a really good way to bounce the ones that should have been killed long ago. Like the -tw extension for Time Warner, used once, spammed incessantly. That error of not using the standard means that I’ll have to host this domain myself until I’m willing to change every one of those extensions.

A catchall email address fell to a dictionary attack where I was getting 3000 pieces of email an hour overnight, and 50,000 before I gave up on the concept.

The final straw was the recent power outage. So now I’ve got a virtual private server running the inbound mail for the domain using a standard debian stable install of postfix, forwarding to Dreamhost to provide user accounts, POP, IMAP, and outbound service if necessary.

There are three important bits of configuration: virtual aliases, control of the recipient delimiter, and an easy way to blacklist the extensions lost to spam.

Virtual aliases are implemented in the standard manner, a set of address destination pairs. All addresses served by this machine are listed here.

To prevent the server from sending to Dreamhost using the – recipient delimiter, the virtual map had to be removed from the propagate_unmatched_extensions parameter.

And finally, the spam victim recipients are culled using the check_recipient_access map, which allows for pairs of the form: user-extension@ REJECT message to be used to reject email to that extension.

A main.cf fragment.

virtual_alias_maps = hash:/etc/postfix/virtual

# prevent dreamhost from getting the recipient extensions.
propagate_unmatched_extensions = canonical

# recipient filters. 
smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/client_access, 
     permit_mynetworks, reject_unauth_destination

Useful configuration links: Propage Extensions, client access map, different bounce messages

The best takeaway here is that when you’re doing something non-standard because it’s better, sometimes the world moves faster than you have time to evolve.

This (Digital-Image Color Spaces) is a good article on the current state of support for specified color spaces in images in browsers. I’m going to be changing a few things around here because of it. First of all, I’m going to be using and specifying the sRGB color space in web sized images, since I think right now I’m at non-color managed and effectively non-specified. Second, I’m probably going to be specifying AdobeRGB for any in-camera jpegs, to capture the wider gamut without losing too much precision.

A List Apart: Super-Easy Blendy Backgrounds

This covers the issues behind png alpha transparency and the workarounds necessary to support IE*

Sometime in the last week or so, flickr updated it’s api and broke my uploader, one that internally used the flickrapi python script. I was getting errors: Error 4: Filesize was zero.

Turns out that the flickrapi.py wasn’t sending a \r\n before the final content separator. It looks like the author figured that out right about the same time that I did, since there’s a new version with a timestamp of yesterday afternoon.

Web Reflection: My DOMContentLoaded

A solution for getting onload handers to fire as early as possible after the page has loaded, but before the images and other whatnot.

There is some info out there on how to use com and activex components from python, but most of the walk throughs and howtos focus on using com to communicate with processes that reside in their own address space and don’t need to have an event loop. A good clue that you’ve got this sort of control is if you get a catastrophic failure exception when calling a method in the dll the first time.

One control that I’ve had to deal with needs to be instantiated into some sort of container that has an event loop. In the .NET world, that means inheriting from an AxHost and instantiating the control on a windows form. In the python world, that means either wxPython or WinPython. Since I’m integrating into an existing wxPython app, that’s my choice.

The first hurdle is figuring out the CLSID and actual name of the control. In WinPython, the tools menu has a makepy.py command. It will pop up a browse list of the human readable names of the COM objects that are installed on the system. that script will generate glue in the site-packages/win32com/gen-py directory, with the filename being the CLSID and version of the COM object. Open that file, in the top page there will be a CLSID formatted correctly, and near the bottom there will be a line that contains the Name of the control. In this case, it’s RANGER.RangerCtrl.1.

With that information, the minimal wxApp that worked with my control was this:

\#!/usr/bin/python

import win32com.client
from wxPython import wx 
import wx.activex
import wx.py

class evt:
  def OnTransportNewState(self, newState=None, oldState=None):
    print "Changing State, %s to %s" %(oldState, newState)
  # ...
class mainwindow(wx.Frame):
  def __init__(self):
    wx.Frame.__init__(self, None , wx.ID_ANY, "Test")

    # turns out that this isn't necessary   
    #self.Show()

    ctrl = wx.activex.ActiveXWindow(frame,
                   clsId=wx.activex.CLSID('{1C5C9095-05A8-11D4-9AF9-00104B23E2B1}'),
                   id=-1,
                   name="somename"
                   )

    # the dispatch to send messages to the control
    self.d = win32com.client.Dispatch("RANGER.RangerCtrl.1")
    # Event object with call backs to receive events
    self.e = win32com.client.WithEvents(self.d, evt)

    # this lets you poke around
    self.shell = wx.py.shell.ShellFrame()
    self.shell.Show()
    # and this shouldn't fail badly, at least for this control
    print self.d.GetVersion()

if __name__== '__main__':
  app = wx.PySimpleApp()

  frame = mainwindow()
  app.MainLoop()    

The frame doesn’t need to be shown on screen, but it does need to be created to receive events. This control uses call backs to intercept events, so I’m using an eventbuilder dispatch class to catch those. Both the dispatch object and the ActiveX control are necessary, as it doesn’t appear that the activex control actually gets the methods that are supposed to be included. I’m not sure if this is actually common over controls of this sort, or if it’s a peculiarity of this particular control and it’s structure.

The handlers in the event class can be determined by looking in the makepy.py generated class. In this case, there was a list of about 20, of which I’ve intercepted about half.

The final wrinkle in this excercise is packaging using py2exe. Win32com needs to be added as a package in the setup.py, the MSVCP71.dll C++ runtime needs to be included in the base application directory, and wx.activex is really picky about the working directory when it’s imported. Even when that runtime dll is in the same directory as the C runtime, there are import errors unless I’ve chdir’d into that directory. Since this happend on import, it’s something that needs to happen really early in the application.

In the setup.py script for py2exe, I’ve subclassed the build_exe class following the lead of this page. I’m including the following method:

def plat_finalize(self, modules, py_files, extensions, dlls):
  BuildExe.plat_finalize(self, modules, py_files, extensions, dlls)
  # from http://www.py2exe.org/index.cgi/TixSetup
  cpdll = os.path.join(sys.prefix, 'MSVCP71.dll')
  dlls.add(cpdll)
  self.dlls_in_exedir.append( 'msvcp71.dll')

And finally, before I import wx.activex,

if hasattr(sys, 'frozen'):
  ### when running from py2exe and we're run from a shortcut, then
  ### when we import the wx.activex, we need to be in the same directory
  ### as the msvcp71.dll
  if os.path.dirname(sys.executable) != os.path.abspath(os.path.curdir):
    os.chdir(os.path.dirname(sys.executable))

Theocacao: Convert an NSImage to CIImage

I should see if this is useful in the context of my resizer, since I remember that there are some interesting gymnastics that I needed to go through to make it work.

So in preparation for the upcoming Mindcamp, I wanted to glue together the code I have to upload to flickr and Apple’s folder actions to make PhotoBooth automatically upload pictures to Flickr and put them in a PhotoBooth set.

To keep the applescript as small and troublefree as possible, I took an existing applescript folder action (in this case, copy to jpeg) and hacked it up to call a python shell script where I make a few calls using the FlickrAPI module.

-- the list of file types which will be processed 
-- eg: {"PICT", "JPEG", "TIFF", "GIFf"} 
property type_list : {"JPEG"}
-- since file types are optional in Mac OS X, 
-- check the name extension if there is no file type 
-- NOTE: do not use periods (.) with the items in the name extensions list 
-- eg: {"txt", "text"}, NOT: {".txt", ".text"} 
property extension_list : {"jpg", "jpeg"}


on adding folder items to this_folder after receiving these_items
  try
    repeat with i from 1 to number of items in these_items
      set this_item to item i of these_items
      set the item_info to the info for this_item
      if (alias of the item_info is false and the file type 
          of the item_info is in the type_list) or 
          (the name extension of the item_info is in 
          the extension_list) then
        
        set theCmd to "/Users/erics/bin/pb_flickr.py " 
          & (quoted form of POSIX path of 
          (this_item as string)) & "&"
        --display dialog (theCmd as string)
        do shell script theCmd
      end if
    end repeat
  on error error_message number error_number
    if the error_number is not -128 then
      tell application "Finder"
        activate
        display dialog error_message buttons 
          {"Cancel"} default button 1 giving up after 120
      end tell
    end if
  end try
end adding folder items to

The python code is pretty straight forward, the only real tricks are getting api keys, secrets, and tokens. I’m not sure I remember what I did to get those, since I did it a year ago.

#!/usr/local/bin/python

import os
import sys
from flickrapi import FlickrAPI

set_id='72157594368688426' # your set id here...

class flickrprefs:
  api_key = 'your key here'
  api_secret = 'your secret herer'
  auth_token = 'your token here'

if __name__=='__main__':
  argv = sys.argv
  if len(argv) < 2:
    sys.exit(0)
  
  pth = argv[1]
  
  prefs = flickrprefs()
  try:
    fapi = FlickrAPI(prefs.api_key, prefs.api_secret)
    rsp = fapi.auth_checkToken(api_key=prefs.api_key,
                               auth_token=prefs.auth_token)  
    
    if not rsp:
      #token isn't valid.
      sys.exit(0)
      
    rsp = fapi.upload(filename = pth, 
                      description = '', 
                      is_public="1", 
                      api_key= prefs.api_key,
                      auth_token = prefs.auth_token,
                      tags='photobooth'
                      )
    
    rsp = fapi.photosets_addPhoto(auth_token=prefs.auth_token, 
                                   api_key= prefs.api_key,
                                   photoset_id=set_id,
                                   photo_id=rsp.photoid[0].elementText)
                
  except Exception, msg:
    #print msg
    pass # I don't really care if I'm not connected to the net. 

Finally, it's a simple matter of dropping the applescript in the ~/Library/Scripts/Folder Action Scripts directory and enabling it with the Applescript Folder Actions Setup app.

• Trusted Computing for Mac OS X. The hardware is on my system, there’s just nothing in the software taking advantage of it.
• Event Delegation versus Event Handling Delegate from items in a heirarchy, so that only the root nod needs to handle the events, instead of handling the events in each individual leaf node. deusx is doing this in his xoxo outliner stuff.
• Optimizing Page Load Time There’s a trade off for many small requests vs one larger request. The real question is: how often does gsizipped javascript trip up IE, since Yahoo says that all their class a browsers work with gzipped javascript.
• Animations using javascript but lighter than scriptaculous or some of the other heavy weight libraries.
• No Silver Bullet. Some things will always be hard.

These three scripts are the Manila server side counterparts to the getPics, getShortcuts, and getPosts scripts in the MvManila 2.5 package.

Of these, I’d recommend using the shortcut one if possible, since it is a faithful port of the originals, which are hampered by the one big rpc call to get all the shortcuts in one go.

The picture one creates the same file as the original, but does not extract the binary images. I’m using static image hosting, so there really was no need to extract them from the database. This might not work properly for you if you’re not using static image serving.

And finally, the getPosts. It’s the least faithful, and if you’ve got a newer Manila site, you may not need it. I skipped all news items, since in my site they were a really funky calendar of upcoming events. Comments are rare on my site, so all the commenter’s names became ‘Commenter’, but their emails were preserved for the admin. If you need it, it’s probably a good start, but it’s assembly required. I needed this because quite a lot of my posts just didn’t make it through the xml-rpc interface.