Selling online

I was trying to buy something online the other day and it was instructive.

The Good stuff:
The Website was visually interesting and distinctive, and they'd found a product niche where they had little competition. The tone was casual and genuine, and the personality of the business and its people came through very clearly. They'd made good use of graphical elements and telling their story to mark out a very distinct little plot of the Internet. They even had good word of mouth, since that's how I found them in the first place. Their prices were good and they had a good array of payment options - I love to see options like Discover, Amex, and Paypal. All awesome.

The not good stuff:
The form to enter my "Identity theft" data wasn't secure (no https, lots of browser warnings about insecure transport and eavesdropping)

The credit card validation kept failing with a generic error. I kept re-typing the number, address and verification code and it just kept saying "enter a valid credit card". By the third time I'd very carefully entered all my data and it failed again, I became convinced that all I was doing was sending my CC and Identity theft info off to hackers in Malaysia, or somewhere. I became certain that it the bank had already blocked it because the hackers had hit my credit limit. I became convinced it had already been used to purchase nuclear materials in the middle East. I was already resigning myself to a two-year stint of trying to track down and correct all the damage done by my identity theft. This is not the emotional experience you want me to have at your online store. It turns out it failed unless I checked the "save my card in my profile" box. Which just added to my uneasiness. This guy can't even validate a credit card correctly and now he wants my permission to save all this data?

I'm sitting here typing this now and I'm kind of wondering why I persisted.

I got the expected "Thanks for your order - we'll email you a confirmation" page, but a confirmatory email didn't arrive. For days. Yes, I checked my spam folder (though I have yet to see Gmail blow it like that). Eventually, I went back to check on my order (because I'd had to save my Credit Card data, I now had a profile), and my order was just sitting there with a status of "Pending". Pending what? Pending you running my credit card up to the limit? Pending a delivery from the wholesaler? Pending whenever we get around to it? Is there a way to inquire about the order? No contact information on the order status page. No email form, to either the business or the webmaster. No way to cancel the order either, which is kind of a leech tactic.

There's a "customer service" link which has a FAQ behind it that claims they ship all orders received before 12PM on the same day. Well, I know that's a lie. At least there's a snail mail address at the bottom of the FAQ. No phone number though. Here's the very best part: On the customer service FAQ, there's a big block of reassuring text, but it's "loren ipsum" Literally. I'm Sooooooo happy I let you keep my credit card information.

Finally I googled them based on the business name and their snail mail address and got a phone number. It turned out to be one of the other businesses this guy was running. These days we understand about multiple streams of income, so that wasn't that disconcerting, once the receptionist and I worked through it. But it was definitely a shame that he didn't cross-refer between the websites of all his (fairly closely related) businesses. Anyway, I left a message. Let's see what happens.

A Smooth Drive

They've repaved Lomas.

I drive it twice a day, to and from work. Alone, now.
It's nice and smooth, now.

I remember trying to avoid all the bumps it used to have.
Trying to swerve around them and avoid the discomfort.


Driving right into the rising sun
On the way home. I'm tired.

Don't hit the bumps.
Ignore the lanes. Too early for traffic. Or Cops.

But smoothly! Don't jerk the wheel!
Just one more thing he doesn't need.

Damn, I hit that one. Didn't even see it.
Manhole cover. Edge right.
No way to miss that one. Unfair. Rotten.

How little I can do. How little


Hearing those little sounds.
The grunt, the sharp breath of pain.


Chemotherapy

Programmers and firewalls

OK, this one's going to be nerdy. If you're not a programmer, you'll want to surf elsewhere. If you're a programmer, please read and learn. If you're a network engineer or firewall admin, read and sympathise.

Executive summary:

Don't put your programs on weird TCP (or UDP) ports. Just don't. It doesn't help your security at all, and it blocks you from a bunch of legitimate users.

End Executive Summary



So I'm currently implementing a multi-site firewall and VPN installation, and it's pushing one of my pet peeve buttons hard.

First, Background:
One of the things I strongly support is egress filtering. Many or most firewall admins don't do this, and allow anybody inside out on any port at any time for any reason. That's fine until one of the "protected" machines gets compromised (and they will), at which point it has nothing preventing it from phoning home to the cracker that owned it and joining his botnet, to be used for future evil. Egress filtering at reduces this a little (he has to use an open port, not his "special" one), and if you combine it with some pattern matching, you might catch compromised boxes sooner (hey, the server at tweek.example.net just started generating a bunch of outbound traffic we've never seen before, what's up with that????")

So I do egress filtering. It's no silver bullet, but I think it helps.


The week after you bring up a new firewall, you always have to spend a few days punching additional holes in it for things you either didn't think of or didn't know about.

(Punching holes in the firewall is bad, MMMMkay?)

Stuff you didn't think of is on you. Bad Network Engineer! No Tee Shirt! Unpaid overtime instead!

Stuff you didn't know about is often (6 cases so far, and counting....) the fault of some programmer that got lazy or was ignorant and decided to put their application on a nonstandard TCP port. Case in point, I've discovered a webserver outside my firewall that goes along fine (on 80/443) until you do a search, and then the URL jumps to "http://lame.server.example:3442/rest/of/url"

Whoever wrote the search took it off 80/443 and put it on a nonstandard port. Just because.

There are two reasons I can think of why this turns out this way:

Laziness: Progammer doesn't want to take the time to learn and implement this in a way that integrates it with apache or IIS or whatever. No excuse for this. You don't want to be this programmer, you'll be re-inventing the wheel. Insecurely.

Ignorance / false sense of security: The programmer thinks "webservers get compromised all the time (True), I'll stick my listener on a weird port where no one will find me" (False). You don't want to be this programmer either.

I used "False sense of security" pretty deliberately. Twice now. Here's the deal: Any attacker with any chops at all, which is to say any attacker you should be worried about, is going to scan your box for open ports. Even if he weren't going to scan you (and he will) don't forget that "http://lame.server.example:3442/rest/of/url" is publicly available in the hyperlinks throughout the rest of your website, and that big fat :3442 hanging out there looks like a "hack me here" sign. So your hidden port is anything but, and of course your code is less likely to be both examined and updated for security fixes than Apache's. Or Microsoft's. Or anybody's. You might lose a few anklebiters, but only the ones too lame to download nmap. All the ones you should be worried about will be right there, Hackin ur box on ur nonstandard portz.

Worse, you're making your life (and the lives of a bunch of well, not innocent, but.... otherwise uninvolved... firewall and network admins) a hell, forever. You'll have to answer endless questions about why it doesn't work and what the app does and why it's off on a weird port. Not the sort of thing you want to do with your day. The network/firewall guy at your office will have to poke a hole for your app inbound, and that'll probably expose machines next to yours, because network people hate doing things for single hosts (that's an easy way to slow the firewall to a crawl). Are you sure you didn't pick a random port that's being used for some crummy code that's full of exploits and being hammered on constantly? Are you willing to BE sure? Every day??? Even if your code is perfect (hah!), you don't need all this exploit traffic hammering on you all day every day. Plus you just talked your firewall admin into exposing the machine next to yours, which may be vulnerable.

Worst of all, at the other end of the Internet are.... Your customers. Lots of them (more every day) have firewalls and egress filters and can't reach you because their firewall doesn't let them out on 3442. You're essentially limiting your customer base to people who are willing to poke holes in their firewalls for little or no reason. If the remote firewall admin accommodates programmers like you, he'll end up with a long, complex, slow rule base that's full of holes. If he doesn't, then his users just don't get to use your app. There ought to be a joke here about limiting your customer base to newbs and lame-os but I'm too tired and pissed to think it up.

So it's the worst of both worlds. You don't lose any of the evil intruders, and you do end up losing a lot of legitimate users. Plus you're pissing off your firewall admin. He can cut off your Internet, don't you know? Or even replace your internet with kittenwar

Take a lesson from the crackers. Put everything on 80 and 443(*). You'll fly through all the firewalls in the world, and no one will ever know, and your application will just WORK and you won't have to spend your time answering questions and writing up how-tos on getting around the corporate firewall.

Plus, though there'll be no money, on your deathbed you'll receive total consciousness. So you'll have that going for you. Which is nice...

(*)No, it doesn't actually have to be web traffic. It'll look like web traffic to yobs like me, and fly through. Which is what you want.

Bitcoin and Internet security

For those of you who don't know (I didn't until recently), there are a bunch of nerds who are trying to create a cryptographically secure, anonymous, distributed Internet-based currency called BitCoin. For more information about Bitcoin, here are a couple of podcasts (one long and one longer).

For an amusing and informative fiction piece on digital currencies, please read and enjoy Neal Stephenson's "The Great Simoleon Caper"

Within the last few days, Bitcoin has suffered a major setback - someone hacked an exchange (where bitcoins can be exchanged for other currencies), and used large volume buy and sell orders to steal a bunch of money (presumably Dollars or Euro or Yen) and then drive the exchange rate for bitcoins to almost zero.

The first thought I had was that apparently Bitcoins are now officially at least worth stealing. And even if this sinks BitCoin beneath the waves, the open source code can go on to inspire and inform other efforts, so there's no way to put that genie back in the bottle.

Here are other some details I found interesting.

Bitcoin itself was allegedly not compromised. People's Wallet accounts at a popular (Mt.Gox, the most popular) bitcoin exchange were compromised. Apparently not all accounts at Mt.Gox were compromised.

Even the exchange was not hacked directly - apparently a copy of the encrypted password database held by the auditors got loose into the world and was used to launch the attack. It's not clear how the auditor's copy got outside *their* office network, but the lesson is that your security perimeter is almost certainly bigger than you think, and there are edges that are very difficult to watch. There is no setting on your corporate firewall that will protect a file that's at your auditors' offices.

The attack seems to have been a password discovery attack -- the attacker has a copy of everyone's passwords, but they are encrypted. The attacker runs the (known) encryption algorithm against either a dictionary of likely passwords (a semi brute force attack) or against a file of all possible passwords (a massively brute force attack) and see if any matches pop out of the encryption algorithm. If so, any account where a match is found is compromised. The variables are the quality of the encryption algorithm, the strength of the password, and the amount of time and compute resources that the forces of evil can devote to the attack.

Of these three variables, the one most directly under the exchange's control (besides not giving their auditors a copy of the password file) is the password algorithm. They had recently upgraded the algorithm, but some accounts that hadn't been logged into recently still had the old algorithm and were more vulnerable. In this case they did the right thing, but it took too long.

The factor most under the users' control was the strength (and freshness) of the password. If your password is "password", that'll be cracked in no time flat, as "password", "Password". "PassWord". "PASSWORD", and "P4ssw0rd" are probably the first five entries in the crackers' dictionary of possible passwords. In fact, if your password is this poor, a cracker doesn't even need an offline copy of the encrypted passwords. He can log directly into your live account with only 3 or 4 failures, which nobody is going to notice.

The big mistakes I saw reported were, first, letting the auditors have a copy of the encrypted password table (financial auditors don't need this at all, and data security auditors ought to work with it on site only if at all possible, and destroy any copies after the audit), and, second, the fact that that copy got out of the auditors' control and into the world.

The luxury of having a copy of the encrypted passwords, and being able to attack it in secret in the volcano lair of the bad guys allows them to bring vastly more resources to bear on the problem and prevents any notification that a password compromise is being attempted. If they'd been bouncing their millions of incorrect password attempts against the live authentication server, the resulting large number of login failures might have been noticed before any compromise was achieved, and they almost certainly would have activated any account lock-out mechanisms in place to foil just such an attack.


Lessons for the ordinary user are:

1. Only use strong passwords. This is so critically important that I'll devote a post to it ASAP, but in the mean time google "strong passwords" and review and learn...

2. Change your passwords periodically. Your bank may have mistakenly given a copy of your encrypted password to their auditors, just as bitcoin did. Assume it takes three months for that to get out and for your password to be compromised by the forces of evil. If you have changed your password in the interim, your bitcoins (or dollars) will be safe while others are compromised.

3. Don't use the same password for multiple sites, particularly where the risk factor is high. If someone breaks the password you used to use for the local dialup bbs account you haven't touched in three years, you don't care.... Unless you're using the same password for your bank.

This is all hard. Good security always is. There's no way you can implement this and still have a hope of remembering all these different, current, unique, difficult passwords, so all I'm going to say is: Password safe. Encrypted. With a darn strong password. That you absolutely will NOT forget.

My personal fave is KeyPass, but there are others, and I haven't seen a code audit of KeyPass anyway (not that you should trust my opinion even if I had).

Be safe.

Gerswin and Dvorak

Tonight I attended the NMSO's Dvorak and Gershwin concert.
Dvořák, Symphony No. 8
Gershwin, Concerto in F
Gershwin, Rhapsody in Blue

I like both composers, and had a very good time. My feeling about music, particularly symphonic music, is that the composer's job to start with a theme, and use that theme to transport you to different environments and emotions. Both of these guys can do that just fine, thank you.

However, it was interesting how differently the two composers made me feel. Dvorak took his theme and at different points, I was watching a butterfly in a pastoral park, with beautiful, flawless landscaping. I felt I was riding through a park, on an immaculately groomed horse. I was transported to a Victorian era party, with waltzes and bowing and hoop skirts. Dvorak built a structure of grandeur and heroism that ultimately started feeling martial to me. It's not so much individual heroism as it was the Russian army on the march. Dvorak took me there very smoothly and with gentility, at every step of the way it was like being in the hands of a great concierge that just takes care of everything.

Gershwin is so much more rustic and raw, and ultimately to my ear, American. I realize that there are those who think this is unfortunate, but I have to tell you that I enjoy it immensely. With Gershwin you sort of wake up in a jazz club in Harlem with no recollection of how you got there. Then suddenly you're at a rodeo or a hoedown (It's fun to watch all those violinists strumming away). Gershwin does the horse ride too, but it's out in the wilds or the countryside, not in a manicured park. When Gershwin goes for grandeur, it's more like the Grand Canyon, raw and elemental and rugged. When Gershwin brings in a march, it's not the Russian infantry, it's a high school band in a Thanksgiving parade. He makes a lot of sharp turns with no warning, one second you're in downtown Manhattan and then you're suddenly on an elephant in a circus parade. The music will be bright and celebratory, then suddenly stop and a musical tornado suddenly rolls over the plains and there is thunder and flood and looming clouds.

And of course, when Gershwin finally brings the different threads of this whirlwind tour together, and puts the whole orchestra behind it, and the beautiful, passionate, longing flower of the piece finally blooms, briefly, and twice, it's more than just a musical orgasm, it's like falling in love. Gershwin. American. brilliant. heartbreaking.

The NMSO is savvy, and does this every year. Go.

Straw or no straw

OK, so I frequently drink soda, tea and water, all with ice.

When I drink soda, I MUST have a straw.

When I drink water, I SOMETIMES use a straw.

When I drink tea, I NEVER use a straw.


Clearly it's not about ice cube management...

Double opt-in: what is it, and why do I care?

OK, so in my last post I mentioned I was happy to run into a case of double opt-in. I wanted to be clear that this was basically the ONLY thing this Internet marketing effort did RIGHT, and explain why.

The point of all marketing is to get someone's attention, create a positive impression, and to attempt to leverage that into at least one sale (preferably a lifetime of sales). That's your mission, that's your ONLY mission, anything that interferes with that is to be avoided. Double opt-in is your best tool for that.

First, let's review opt-out. Opt-out is when you're included in something (in this case, an emailed marketing list) without any action on your part. This is your (landline) phone company's strategy: They sell (yes, they make money from it) your phone number to marketing companies and then these same marketers call you at dinnertime and try to get you to use a new phone company (who will then do the same...). The beauty of the phone company's model is they get paid by both ends: By the marketer that wants to annoy you, and by you so you can have a phone which marketers will use to annoy you. If you get an unlisted number, they won't sell it, but that costs you more every month. The Phone Company is frequently evil and this is lucrative for them, but that's not the point.

The point is you have to take action to get OFF the list (you opt "out" of the list), such as getting an unlisted number or requesting the marketer to take you off the list, or pressing "3" (or whatever) when the recording offers the opt-out option, or by contacting the national do-not-call registry. It's very annoying in telephone-land, on the Internet this means 95% of all email was spam at one point (maybe worse now). Compared to telephone operator time (or even telephone operator robot time), email is stupidly cheap, so the problem is much worse, as anyone who's had an email address for a couple of years can tell you (especially if your ISP sucks at Spam Filtering).

So, for the person trying to do Internet marketing successfully, this is just an expensive way to get ignored: You're either spam-filtered or at best you become one of thousands of messages that your potential customer has to sweep out of her inbox before she can get to email she cares about. If you're noticed at all, it'll be negative attention, generating a rotten first impression and if she remembers you at all, it'll be as that dude that sent her all those spam messages that she discarded along with the ads for porn and cheap viagra. You don't want to keep that kind of company in the customer's head, so don't use opt-out, ever. By the way, if the shopping cart page on your Internet store has a prechecked "contact me about stuff" option on the page that the customer must uncheck, then you're doing opt-out, it's just a "fine print notification" type of opt-out, and should be ashamed of yourself (and your marketing efforts will ultimately be expensive and ineffective, which serves you right).

Opt-in is better, but it's far from good enough. You have to take some action to get on the list, but on the anonymous Internet, this bar is pretty low. The classic example is "enter your email address to get our newsletter". The problem is that on the Internet, no one can assume you're you. There's no way to prevent a trickster from entering anybody's email address into the page and hitting submit. Some miscreants (not me, I swear) think this is a funny trick. It's the Internet equivalent of signing your mean neighbor up for lots of magazines and catalogs. Again, the costs are lower on the Internet, which means the volumes are higher, and you're back in amongst the pornagra ads, getting no or negative attention, and therefore wasting all your Internet marketing budget. So don't use opt-in, ever.

Confirmed opt-in is even better: The customer must enter their email address into a subscription page, and you immediately send him a confirmatory email that says "Somebody (hopefully you) entered your email into our subscription page, and you're now receiving our newsletter. If this was a mistake, or you've changed your mind, click *HERE* to unsubscribe". Again, you'd think that this is enough, but it's not. The Anonymous factor on the Internet strikes again. Just as you didn't know whether the customer was really booyah@whateveremail.net, he doesn't know if this confirmation is really from Widgets, inc. Maybe you're just an evil spammer who is using a fake unsubscribe link to confirm that booyah@whateveremail.net is a real email address with a real, actual, person who can read behind it. Evil spammers turn around and sell lists of confirmed email addresses for MORE, and the result is more Pornagra ads. So we've all learned the hard way not to click on links that may be untrustworthy, just like we've learned not to eat candy off the sidewalk. It's much safer for your potential customer to just avoid you, either by abandoning their email address entirely, or by training their spam filter to automatically ignore you. Result: 100% of what it costs to do this is wasted.

Double opt-in is another step up, and is the only way to go. Here's how it works: your subscription web page says up front in big print "we use double opt-in for your comfort and convenience", and offers a place for the customer to enter their email address, along with whatever explanatory text you need to explain and reassure the customer that you're taking their trust in you seriously. When they hit "submit", the ONLY thing they will ever get is an email sent to that address that says "we want to be sure you REALLY want our newsletter. We use double opt-in to prevent pranksters from signing people up who don't want our newsletter. If you *don't* want to be on our newsletter mailing list, do NOTHING, and you *won't* be added to our list. If you are really you, and if you really do want our newsletter (which will come about 2-3 times/month), then click this link to start your subscription." The newsletter will only go to the customer if they request it TWICE. That's how it works. Before we talk about why it's the ONLY way that REALLY works, let's address a common objection:

"It's too many hoops for the customer to jump through". No, it's not. Anybody who can't follow this set of instructions won't be able to order from your e-commerce site anyway. Anybody who WON'T follow this set of instructions doesn't really want your newsletter that badly at all, and would just ignore it if they got it. You want everyone on your mailing list to eagerly devour your insightful, amusing, and informative newsletter, and order everything on it. The person who got half way through the double opt-in and gave up isn't this person, and will never be. You're wasting a 100th of a cent every time you send them an email, and this adds up.


Why it works:

This is one way you can show the customer that their trust is important to you. There's no way for you to smile and offer positive body language over the Internet (yet, they're working on it), so humans have come up with other signalling mechanisms. One of these is double opt-in. It says Your email address is valuable to me, and I will be trustworthy, and I want you to SEE me making an effort to be trustworthy, because I want there to be no doubt in your mind.

This is also a way you can reinforce the fact that your newsletter isn't just a bunch of lame ads (OH, BTW, your newsletter has to be more than just a bunch of lame ads. More about that later). Reinforce the fact that your newsletter is itself a valuable interaction for your customer, full of insight, humor, and valuable information.

Finally, this is the ONLY way to make sure (double sure) that your newsletter mailing list has ONLY addresses on it which represent real people who are really interested in what you have to offer. Emailing THESE people will be effective, will lead to sales, will prevent wasted marketing expenses. Emailing anybody else is a waste of your advertising budget. Don't waste your resources. Don't end up with a reputation as a Pornagra spam artist. Don't make a bad first impression. Don't use anything less than double opt-in.

I ought to talk more about how to make SURE anybody doing Internet marketing is REALLY doing double opt-in, but I've rambled on long enough.