Failing Forward

In a previous post, I talked about how some email clients will mangle the display of your name (“on behalf of” yourself) when you use Gmail as a client for other addresses.  This is because, when using Gmail as a hub for other email accounts, when you send email via SMTP, it’s Gmail sending the mail, not your mailserver du jour.  And so, to avoid the message looking like spam, Gmail tacks on some extra headers.  In certain clients (such as Microsoft Outlook), these headers would get displayed in awkward ways.

A post to the Gmail Blog yesterday, however, informs us that this no longer need be the case.  Gmail now features an option to send mail properly via your email hosts SMTP server.

Wonderful – now my personal address won’t be revealed in emails I send for work or school.  To configure the new feature is a breeze, and there are already tons of material on it, so I’m not going to duplicate that content.  However, it’s not so easy if you want to use Gmail as a funnel for a Live/Outlook account.

Back in May, my school migrated from in-house email hosting, to using the Microsoft Live platform.  That’s fine – less strain on the campus network sounds good to me.  Setting up POP access to an Outlook account was easy when the change happened.  The server is just outlook.com, running on port 995 for SSL.  Great.  So shouldn’t the SMTP setup be as simple?

Hah.  “Should.”  My favorite word.

If you’re thinking “set the SMTP host to smtp.outlook.com, and use SSL port 465,” as per the standard practice – you’d be wrong.  No, in fact they do not expose their SMTP hosts directly.  You actually have to log into your Outlook Web Access account using the ugly address https://pod51000.outlook.com/owa, and then once OWA loads, it will redirect you to a new ugly host (for example, pod51015.outlook.com, or something like that).  That host is your SMTP server.  Once you jump through the flaming hoop… no SSL.  Only TLS, which Gmail doesn’t support.  So port 25 is all you get.

Oh well, at least it works.  Hopefully Microsoft will start using a more friendly SMTP system, and roll SSL into the package (or Google will add TLS to the secure connection options).

Update: It seems that, while this solution does work, it’s not a permanent miracle.  Evidently the exact address can change from time to time, necessitating that you repeat this process in the future.

Fantastic idea.

When Apple introduces a new version of their operating system, OSX, they introduce an (singular, lone, one, uno, distinct) operating system.  When Microsoft releases the version du jour of their operating system, Windows, they release several.  Like half a dozen.  I ask the following question out of real honest to goodness curiosity:

Why?

What are the pros and cons of offering such a confusingly high number of different “versions” (read: software entirely capable of the exact same thing, but throttled with a licence key)?  What are the pros and cons of Apple’s draconian trend of spoon feeding the consumer exactly one way of doing things?

Quite a while ago, I found a really useful way to use C-style comment blocks to quickly enable and disable pieces of code.

Let’s say I have a loop that I want to toggle on and off for testing purposes.

for ( int i=0; i < 100; ++i ) {
  do_something_cool( i, 2*i );
}

There are two ways (and I’ll show you a third) to comment this code.  Both of which you already know.  This

/*for ( int i=0; i < 100; ++i ) {
  do_something_cool( i, 2*i );
} */

and this

//for ( int i=0; i < 100; ++i ) {
//  do_something_cool( i, 2*i );
//}

Not very exciting.  What's more, with long comment blocks, the former can be cumbersome, and the latter... well... O(n) isn't fantastic time-efficiency when you're typing two slashes on every single line of code.  The new method is just as quick to implement as a C-style comment, but far easier to toggle:

//*
for ( int i=0; i < 100; ++i ) {
  do_something_cool( i, 2*i );
} //*/

Just remove the very first slash will turn the comment block "on":

/*
for ( int i=0; i < 100; ++i ) {
  do_something_cool( i, 2*i );
} //*/

Re-insertiing the slash will disable the comment (turn it "off') again. Of course, loads of editors and IDEs have a button to comment/uncomment code with the click of a button once it's highlighted, but for quick toggling, I've found this method even faster. Enjoy.

Often, all a developer needs to solve a problem is to hear himself or herself state said problem out loud, and the answer (which had previously plagued them) seems evident with obnoxious clarity.  I had a similar moment just now.  I was about to post the following question on Stack Overflow.  I encourage you to follow along, and try to solve the problem with me.  You’ll see where I had my epiphany and stopped typing.  All I had to do was state the problem.  My great friend Frank and I often do this to one another on a very regular basis.  Here’s the (one sided) conversation:  “So, I’m having this problem where [fill in the first 70% of the explanation here]… Oh.  Never mind.  Duh.”  For this reason, I’d highly encourage every developer to find, and maintain a tight relationship with, an exit buddy coding buddy.

Here’s the contents of the question I nearly posted:

Premise: I have a MySQL table defined as follows:

CREATE TABLE IF NOT EXISTS `stuff` (
    `id` int(10) unsigned NOT NULL auto_increment,
    `hash` varchar(255) NOT NULL,
    `field1` varchar(255) NOT NULL,
    `field2` varchar(255) NOT NULL,
    `field3` varchar(255) NOT NULL,
    PRIMARY KEY  (`id`),
    KEY `hash` (`hash`)
);

This table can easily grow to a few million rows or more.  Every record has a ‘hash’ value which is the result of running some of the data fields through SHA1().  In practice, I see very high rates of duplication here.  Hundreds of thousands of records are loaded into this table using ‘LOAD DATA LOCAL INFILE’ queries on a nightly basis.

I implemented this solution for ensuring all duplicate records were removed, in the form:

DELETE stuff
    FROM stuff
    LEFT OUTER JOIN (
    SELECT MIN(id) AS id
        FROM stuff
        GROUP BY hash
    ) AS keepers ON (stuff.id = keepers.id )
    WHERE keepers.id IS NULL
;

I find that after around 6k rows, the complexity of the DELETE query begins to really slow down.  Is there a more time/space efficient solution to achieving the same end?  Or perhaps a different way to phrase my DELETE query so as to preclude certain overheads?  As far as I’m aware, there isn’t a way to combine ‘LOAD DATA LOCAL INFILE’ with ‘INSERT IGNORE’ – otherwise I’d simply

… And that’s when I thought I might just – on a whim (just in case I’m wrong) – check the MySQL documentation.  In other words, “RTFM.”  And what would you know.  All I needed was to state the problem, and the obvious answer smacked me in the face.  In this particular case, my coding buddy was a website (Stack Overflow), but usually it’s another developer.  In any case, you need to have a coding buddy.

If you’re interested in more bulk loading goodness, you can get it straight from the horses mouth.

Update: Evidently, Jeff Atwood seems to agree with me, although his language and arguments are far more graceful.

I got distracted on Stack Overflow again today.  This time, it was reading a thread on the subject Great programming quotes.  Here’s my pick:

A programmer started to cuss
Because getting to sleep was a fuss
As he lay there in bed
Looping ’round in his head
was: while(!asleep()) sheep++;

I thought it was pretty clever (it made me chuckle at least).  And when I read this comment…

I would use “++sheep”, rather than “sheep++”. It will do less copying, which is important because a sheep is a pretty complex object.

I absolutely fell over laughing.  I don’t think any hard core dev would have trouble seeing the epic humor in this logical critique of the poor restless programmers’ toSleep() function.  However, my laughing fit was short-lived.  Because, nearly as quickly as the humor sank in, another feeling began crowding it out: uncomfort.  I didn’t quite know why.  I read the comment again, and instantly knew why my Spidey sense had been tingling.

sheep, as presented, would clearly be some form of integer.  Actually, unsigned integer (if you don’t want to run certain risks).  And in the case of incrementing* a lone integer (meaning, it isn’t being used as part of a more complex expression) there won’t be much difference here.  However, it’s a good platform off of which to launch a discussion about the semantics between pre- and post-increment operators.

At it’s easiest, here’s the difference:

• Pre-increment (as in ++x) increments the value of x, and then returns the new value, as opposed to
• Post-increment (as in x++) which increments the value of x, but returns it’s old value

Here’s an example:

int x = 4;
int y = x++;

Here, y will wind up with a value of 4, and x will end up holding a 5.  When we use pre-increment though, as in:

int x = 4;
int y = ++x;

Both x and y will hold a value of 5.

The latter method is the faster of the two, and the reason is pretty simple.  In order for the post-increment operator to work, it must first store a temporary copy of the value, increment, and then return the copy.  The pre-increment implementation simply increments and then returns itself.  In that, our comedic commenter was correct – the pre-increment operator can be faster than post-increment.  However most compilers will automatically optimize an increment statement [if they can do so without altering the semantics of the code].  And, in the case of

if ( !asleep() )
    sheep++;

the compiler would most certainly do so.  So, while our amusing annotator had the right idea, unfortunately he was a bit mislead in this case.

Still, I think it’s a riot.

*I use the case of pre- and post-increment here for brevity.  The same mechanics are at work for the pre- and post-decrement operators.

Well… not quite.

This might be an old, tired, boring hack to some, but it’s new to me.  I had a function – call it f()… my math background won’t leave me alone – and I needed it to take several forms.  In one place, I wanted to pass in a single value, and in another place, I wanted f() to operate on an array of such values.  But PHP does not support function overloading because it’s not cool enough.* After a bit of thought, I came up with a pretty sane way of having the cake and eating it, too.

public function f( $x ) {
  if ( !is_array( $x ) ) {
    $x = array( $x );
  }
  foreach ( $x as $item ) {
    // Do your work on each element
  }
  return;
}

Cheers!

*This is actually an artifact of the extreme flexibility of the language.  Because of PHP’s loose, dynamic typing, there isn’t a sane way for PHP to be able to tell the difference between a call to go( $a ) where $a holds an integer and go( $b ) where $b holds a string (read: there isn’t a compelling enough excuse for anyone to waste their time implementing such a feature).

There seems to have been (in the past year) a surge in the market to label any type of Information Retrieval system a “Google Killer.” First it was Cuil (which wasn’t so cool) which was a big flop. Then this weirdo came out with Wolfram Alpha, the “Computational Knowledge Engine” – not as much press as Cuil, so not as much flopping. And now we have Bing (aka “Live Search” 2.0 (aka “MSN Search” 2.0)) which had a fantastic debut, slipped back down, and has been gaining more traction.

“Google Killer” was definitely a would-be appropriate label for Cuil or Bing. But Wolfram Alpha, not so much. The “weirdo” that envisioned it is actually a man by the name of Stephen Wolfram, who contributed to the areas of cellular automata and studying Universal Turing Machines. [Actually, Wolfram did a lot more than just that - but having studied cellular automata in some depth, it's his work with which I'm most familiar.]

Wolfram Alpha, the Computational Knowledge Engine, is apparently just that. While slinking around online, I came across a link from a woman claiming that every living human being could fit safely on the landmass of the Australian continent, with more than a quarter acre to himself or herself. Provided was a link – to a Wolfram Alpha query. I was blown away when I followed through (out of curiosity only). To screenshot the resulting page would be to do Wolfram Alpha a disservice, you’ve got to follow the link and play around with it on your own.

Suffice it to say, I will be using Wolfram Alpha again. Soon. A lot.

It knows geography, it knows population statistics, and it can divide. Some would say “big deal.” I think there’s a bit more to it than that. Whatever your feelings, at least it knows the most important answer of all.

I thought that I might update the blog’s styles a bit to go with the new name, and finally found a good match.

While working today, I set about to expire various passwords to various hosts, accounts, web services, et cetera.  One of the websites we use here is called SiteGround.

They fail.  In my personal or professional use from here on out, I will not be visiting that site again if I can at all avoid it unless they really change their tune in a big way – and here’s why:

There is absolutely no reason I (as a web developer) should ever limit my users’ ability to come up with long, complex, secure passphrases for my website.  Additionally, I see no reason that, as an administrator, I should be asked by a third party to trim my passwords.  I could understand if there were hard limits involved, mind you:

• Maximum input length to *MySQL’s PASSWORD(), MD5(), or SHA() functions
• Limits on the sizes of GET and POST variables

These are two very good reasons for capping the length of any user-submitted data.  But, if you’re familiar with these sorts of things, you know that it’s not likely anyone will be miffed by those caps anyway.  Using a generic (sufficiently random) password generator with all 52 letters (upper case and lower), ten digits (zero through nine inclusive), hyphens and underscores, even just a ten character passphrase is less than one combination in one quintillion.  Yes, that’s “quintillion” – that’s a million trillions.  The only thing I can think of to even come close to analoging the ridiculous odds against anyone brute-forcing a 10 character base 64 passphrase is the following quote from the new **Star Trek movie (which, by the way, I highly recommend):

“The notion of transwarp beaming [brute forcing that password] is like hitting a bullet with a smaller bullet while wearing a blindfold, whilst riding a horse.

I’m no cryptographer, for sure.  I’m just a lowly web developer – but even I know, at around 10 or 20 characters (at base 64), you’re far more likely to be the victim of a social hack than you are of someone compromising the password randomly or programmatically (excepting vulnerabilities in the hashing algorithm itself).  But that’s not the point.  The point is that there is no decent justification for capping user password sizes other than in scenarios similiar to those listed above.  SiteGround’s code monkeys wasted time imposing a maximum string length and generating the above error message.

Have you ever come across similarly pointless restrictions?  Have you yourself been made to enforce things like this?  I’d love to hear about it.

*Insert your favorite DBMS (and it’s suite of hashing/encryping functions) here.

**I feel compelled to disclose that I am not, nor have I ever been, a Star Trek fan.  Until last Friday.

Update: A certain [undisclosed] financial institution is also limiting their passwords – to 12 characters.

Update: Apparently MySpace limits their passwords to 10 alphanumeric characters (found this out through work).