On Convenience

Yet again there was a big Internet kerfuffle regarding digital music and piracy. It’s been at least fifteen years now. What needs to be said has been said. There is however, a side issue of convenience that I think is being misunderstood.

I think David and I agree on at least one point, which is that Emily’s suggestion that buying music isn’t convenient doesn’t really make sense. It’s gotten pretty easy over the last few years to pay for the music that you like.

Jonathan Coulton

You can see how both David and Jonathan easily agree that paying for music is convenient. It takes very little effort to legally purchase music on Amazon or iTunes. The process is pretty seamless and the music is properly organized. Most of the time you click once and an entire album of music is ready to play.

What they are missing at is that convenience isn’t just about the process of getting the music, it’s about the size of the catalog. If you buy music legitimately, you will often have to use many different stores to get all the music you want. A large percentage of that music is unlikely to be legally available on any store, especially foreign music. Look how many people have to keep switching between different iTunes countries to get the music they want. There should just be one iTunes store with all the music from all the countries.

If someone goes to iTunes or Amazon and searches for music, and no result shows up, that is inconvenience. Now they have to go to all the other legal music stores to see if it is available there. Or they have to try to find the official web site to see if it is there. If they just go to Pirate Bay, all the music is in one place. Convenience means always getting the search result you want, and not going on a wild goose chase. When I say all the music, I literally mean all music recorded from every label and every country and every artist throughout all of history. Just one important missing track is an excuse to pirate.

In addition to the size of the library, the ability to buy in bulk is necessary. Piracy sites don’t just have albums and singles, they have discographies. These discographies go way beyond any boxed set sold by a record company. They even include pirate concert recordings. They sometimes include foreign releases of an album if they had different track listings. I can’t go to the Amazon MP3 store and click one button to get every track that a given artist has every so slightly contributed to.

In the article that sparked this discussion Emily talked about how she ripped 11,000 songs. She did this by sitting there and ripping CDs. That’s not all that convenient. Neither is clicking on every single album and track in an online catalog. If you want to sell me some music, sell discographies. One click to get every Zeppelin track ever for $20. Sold!

There’s also the matter of DRM being inconvenient, but that is mostly gone now for music. When people say it’s still inconvenient it is because the catalogs are severely lacking, and there is no way to buy in bulk. If you want to build something that we will come to, build the music library of Alexandria. The Pirate Bay is already almost such a library. You have to meet or beat that to have any chance at competing.

Digital Identity Management

I read this article today, and it made me think of all the people who have false and/or multiple identities on the Internets.

I’m one of the co-founders of Windsoc, and I have several web projects aside from that.  I’ve done quite a bit of consulting in the past, and I still do the odd one today.  I’ve also dabbled in politics, having run for office once and spent many hours on issues that I feel are important, such as youth recreation and city planning.  And I’m a part-time writer of speculative fiction, spending around 0.1 hours a week on it and seeing my literary career advance at the same rate as the snows of Kilimanjaro.  And then there’s that television series I was working on…

So which one of these “hats” do I wear on Twitter?  I worry about annoying followers by talking about the wrong things, so I generally say nothing at all.  My personal blog has always been political, so I don’t feel it makes sense to start talking about startups or technology or programming.  And there are many things I just can’t say on the Windsoc Blog, since I’m not the only one involved in this venture.

My Identity Crisis: Why can’t I be more than one thing on the web?

Like every person, this guy has multiple aspects of his life that are mostly separate. At the very least most people separate family, friends, and work. Many people have even more separate lives. Perhaps they have multiple jobs, multiple separate friend groups, even multiple families. On top of that we now have different lives as members of various communities that have been able to form thanks to the Internet.

People, myself included, tend to act as different people within these different realms. Conversation topics, manners of speech, or even entire personalities can be completely different when someone is with their family as opposed to when they are at work or out with friends. This is both conscious and subconscious. Even if you are most comfortable acting as a drunken clown, you do not want to do that at your job. Yet, you don’t have to think about it. The environment of the workplace changes your behavior such that you act in a certain manner.

On the web, it’s a bit different. You haven’t actually changed your surrounding physical environment, or the physical people that are surrounding you. It’s the same you in the same place with the same Internet connected device. Thus, even when you interact with different groups, you don’t automatically change who you are.

Sometimes you might be interacting through web forums, which is like changing environments. It’s relatively easy to be a different person on Fark than you are on Reddit. It’s much more difficult to be a different person on Twitter, since it all has to be conscious. You have to actively decide how to act based on what user you are logged in as.

There are things you can do to help that subconscious switch. Maybe use a different Twitter client for each different account. That way you can associate the different aspects of yourself with different visual environments. Use the twitter website for your family twitter account and use Tweetdeck for your business tweets.

Even then, I have a better idea. Stop being multiple people. I really don’t like this aspect of human culture that we change who we are. I suffer from it as well as anyone else, but I try to do what I can to notice it and reduce it. Whether it’s family, friends, or work I try to break down the barriers between them.

It might be trying to get my family members to come to a geeky convention, or telling my coworkers about My Little Pony fandom. I want to be my most comfortable and true self all the time. I don’t want to change who I am because of where I am or who is nearby. I want to have one “true” identity that is always on. I want to always be the same person all the time.

And that is why I have just one account in each place. I’ll tweet about New York food trucks, technology, and games all in the same hour. I’ll connect to co-workers, family, and friends all from the same LinkedIn account. I’ll friend my mom on Facebook and also people who listen to my podcast.

If I happen to share something online from one aspect of my life that other people don’t understand, I see that as an opportunity. There’s nothing about myself that I need to hide from anyone I know. If someone judges me based on what they see in other aspects of my life that were previously hidden from them, fuck ’em. There are enough other people who will see those other aspects and bridge the gap.

Thanks to the Internet we have been able to bring people so far apart so much closer together. That is the miracle that has caused this situation in the first place. Why then would we put up barriers at the last line of defense to keep people apart?

No, You Need A Technical Co-Founder

I just read this article suggesting that entrepreneurs should learn to code and start technology-based businesses without technical co-founders.

I went to a Y Combinator Q&A last night and started getting frustrated when the audience almost completely shifted the talk to the problem of finding technical co-founders and how extremely difficult that is. Yes, it is difficult. More difficult, I think, than learning some of that tech yourself, which it sounded like most of these intelligent people weren’t trying to do at all.

The Internet Teaches you Ruby-on-Rails

Now, a lot of the advice in this article is actually quite good. The Internet is the single best resource for learning technology, especially programming. Someone founding a technology business absolutely needs to have a strong technical understanding to avoid complete disaster, even if they won’t write any code at all. And even if people do not have formal educations in technology, I strongly encourage everyone to learn as much as possible on their own. I mostly, though not completely, agree with Rushkoff when he says Program or be Programmed.

That being said, learning to program on your own, especially with a framework like Ruby on Rails, can be incredibly deceptive. Just by reading and following instructions a complete novice can be up and running with a web site in just a few minutes. This is a truly wonderful thing, but it is sort of a lie.

You see, this is wonderful if you just want to create a blog or another normal personal site. If you are creating a business, something which people’s livelihoods will depend upon, then you can’t just have a novice whip it up based on learning from the Internet. That’s beyond excellent for a prototype, but you absolutely can not go live with something like that. Even making the simplest web site there are hundreds of advanced and tricky technological issues to address in the realms of security, stability, performance, etc.

If your business idea is something that you can do using WordPress or Drupal, then surely you do not need a technical co-founder. You can go with off the shelf software that is written by experts. It’s only if you actually have to do some programming yourself that you should make sure that a professional does it. Otherwise, you could be opening up yourself to a world of hurt. Then again, if you don’t mind having an application that will crash and be hacked instantly and brutally, do it yourself.

You have the problem of not being able to find a technical co-founder. Your solution was to do it yourself, and I’ve shot that solution down. What do you do? Well, I’m going to tell you that finding a technical co-founder is really easy, and I will tell you exactly how to get one.

First, realize that your technical co-founder is already employed. There is much more demand for these people than there is supply, so they are always employed. You need to get them away from their jobs. Don’t look at resumes of the unemployed. Look at blogs, StackOverflow accounts, Facebook accounts, Github accounts, etc. Find an employed person and lure them away.

Here’s some advice on luring them away. Don’t be mistaken for a recruiter, that will get you ignored. You must begin a legitimate conversation with the person to keep them from ignoring you. If you are incapable of doing that, just give up.

Secondly, realize that this person has a job right now. They are not going to quit that job and work for you for nothing. They need to be paid. That doesn’t mean stock. That doesn’t mean promises of possible future money. That means money today, and lots of it. That means a salary that is at least what they were making before. If you can’t afford it, don’t even bother. Why would someone risk their livelihood for someone else’s idea? Technical people have their own ideas. If they are going to take such a risk, what do they need you for? They can quit, go home, and build something without your help at any time. The best help you can provide is to pay the rent and put food on the table.

Lastly, make sure your idea isn’t completely moronic. If you lack technical knowledge, it’s very likely that something is wrong with the basic premise of your business idea. If you are a complete technological dolt, good tech people will not work with you. If your idea is fundamentally flawed, technological people will just laugh at you.

Here are some common problems with ideas. One is that the idea is technologically impossible. You want to make a free energy machine? Good luck with that. Another is that your idea is much easier said than done. Oh, you want to make a search engine to compete directly with Google? Good luck with that as well. Another common problem is that your idea may be shady in some way. Technological people typically have very strong sense of right and wrong. If your business idea is in any way close to being illegitimate, most nerds will not touch you with a ten foot pole.

Everyone should learn as much as possible about computers and programming. It is very easy to do with the resources of the Internet at your disposal. However, if you plan on starting a serious business based on technology, you absolutely need someone with experience who will know all the bases that need to be covered. The most you can build yourself in a relatively short time is a house of cards.

The Luddites are not actually the power brokers

I just read this article that gives many examples of people who have given up carrying a cellular phone. It claims that by doing so, these people end up in a position of power because others are forced to work on their time.

Everyone has a cell phone now. There are more than 280 million mobile subscribers in America, according to the Federal Communications Commission. According to a 2005 international study by Advertising Age, 15 percent of Americans have interrupted sex to answer their phones. Even people who are videotaping themselves having sex, like Paris Hilton, stop to answer a call. The call of the wildGoogle, Verizon would exclude mobile Web from rulesT-Mobile looks to lag in offering 4G.

Not having a cell phone is a way of getting the world to run on your time. A lot of powerful people are already on to this. Warren Buffett doesn’t use one. Nor does Mikhail Prokhorov, the 45-year-old Russian billionaire who owns the New Jersey Nets. Tavis Smiley doesn’t own one, either.
The Luddites may actually be power brokers

First of all, almost every example in this article is someone who is already a power broker. People need them more than they need other people. If they carry a phone, of course people are going to be calling them constantly. Even the small town dentist has a position of power over his patients. They aren’t going to switch dentists because he doesn’t have a cellular phone. The premise of the article gets the causation wrong. It’s not luddites that are power brokers, it’s power brokers that are luddites.

Even if you are a power broker, should you really get rid of your cellphone? It’s quite a stupid idea if you ask me. Sure, as head of a business you will be able to control when your customers may call you. But what about your family? If there’s an emergency, and you’re unable to be contacts because you are a luddite? That sure will make you think twice. The simple answer is that you should have a phone on you at all times, at least for emergency purposes. If you don’t want to get calls off the hook from customers and such, simply don’t give out the number. It’s that easy.

For those of us who are not already power brokers, refusing to carry a phone will only get people to work on your time as far as their politeness will stretch. It also only works so far as your friends and family are not very spontaneous. If there are things happening, and you are, or could be involved, you will get left out if you can not be contacted. You’ll be relying on the politeness of others to go out of their way to accommodate your phone refusal.

Personally, I have completely stopped accommodating this. My friends all have phones, but they often don’t answer promptly. Other times, they let the battery run out. If I’m about to go out to eat dinner, and I call someone with an invite, a non-answer is a no answer. If they complain later, it’s their fault for not dealing with their phone properly. If I’m waiting for someone, they’re late, and they don’t respond to calls or SMS, I’ll just leave. When they complain later, I can tell them I waited and called, and they didn’t pick up their phone.

There really is no excuse to not carrying a cellular phone if you can afford it. It should be charged, and within close proximity at all times. You don’t have to always pick it up, but at least look at the caller ID. You might say you would never pick it up during sex, but what if it’s your mom calling at 3AM? That’s almost certainly going to be an emergency situation. You will regret not answering. There’s always something more important.

There’s also the situation where people are depending on you. Take for example the superintendent of my building. Yes, I’m paying him indirectly, but he definitely has a position of power over me. If he didn’t carry a cellphone, and I needed him, I would have to work on his time. However, one time the lady who lives across from me locked herself out of her apartment, and it was almost time for her to take her pills, which were inside. My super does carry and answer his phone, but imagine if he didn’t. By being a luddite you become a great burden to the people around you. You need to already be in a powerful social position in order for people to be willing to accept your burden. The boss of the company can get away with it, but only because his assistant does not.

Carry a phone. Keep it on whenever possible (not theaters and such). Keep it charged. Only give the number to people who you want to be able to contact you at any time. You don’t need to answer every call, but check the ID to make sure you want to hang up before you actually hang up. If you follow these rules you can get the best of both worlds. You won’t have a beeping phone ruining your life with interruptions. You won’t be a burden to those people who really need you. And most of all, you won’t risk being out of touch at those critical moments you will regret missing.

MySQL is Great, Haters Gonna Hate

I made a mistake that I must correct. I stupidly broke my rule of only writing comments here, and made a couple comments on this post complaining about a bunch of problems in MySQL.

After being a happy PostgreSQL user for years, I’ve finally had to use MySQL for the last 6+ months.

I have to admit, I was starting to think it wasn’t all that bad–that the “MySQL is a toy” line was outdated.

It’s not.

Dammit, MySQL

I’ve been developing database applications, mostly for the web, for seven years now, professionally for five. I’ve used every kind of database for every kind of app. I’ve used MySQL, Postgres, sqlite, CouchDB, and some others. I’ve also worked on applications ranging in size from a tiny blog nobody reads to gigantic high traffic applications with millions of rows, or millions of hits.

Lately I have noticed that there is a very strong backlash against MySQL. That original blog post was linked from Hacker News, and if you look at the comments there, everyone who is in favor of MySQL is getting downvoted.

Let me tell you something. All these technological complaints people have with MySQL, they are 100% true. MySQL has no transactions. Its foreign key constraints are not up to par. It doesn’t follow standards perfectly. There are very real possibilities of your data becoming invalid, or even corrupt. MySQL is far from being a super perfect database under the hood.

If all those things are true, then MySQL sucks, and we should never use it, right? Wrong. Even though MySQL has these issues, they simply do not matter for the overwhelming majority of applications. If you’re working for a bank, a hospital, or the military, then MySQL is right out. In those cases, reliability and data integrity are so important, and the system is so large, you just can’t be using MySQL. I’m willing to bet you’re not doing any of those things, are you? You’re doing a blog, or maybe a social network.

You might say that you are doing a blog, but PostgreSQL and MySQL are both free and open source. You may not absolutely need the data integrity and standards compliance, but why not have it anyway? It won’t hurt, right?

I disagree with this. In my many years of using both Postgres and MySQL, my experience is that Postgres is extremely finnicky. Because it demands standards compliance and data integrity, it demands a lot more work from the application developer. Let me give some examples I have encountered in the real world.

Transactions are a big reason Postgres is so great. You can make sure your data is always correct because a group of writes will go in all together. This is very important in a serious application. The thing is, if you don’t need transactions, using them can cause you a great deal of trouble. In building multiple Django applications on Postgres, I have seen a very frequent error when a transaction has been opened, but not closed. The result is that the entire database is stuck. You can’t read or write from it until the transaction is closed out. Whenever this happens, I just restart the db server, and everything is back to normal.

It shouldn’t be possible for this to happen! The fact that restarting the database fixes it is proof that it doesn’t even need to happen. Yes, transactions are good, but a small user error causing the entire database to go out of service is much worse. In a huge application, you can take the time to test and make sure you don’t make those errors. For a normal every day application, you just want it to stay up. You don’t care about some unclosed transaction. Just let it slide.

A similar situation happens with the much lauded foreign key constraints. I’ll have myself two tables with a foreign key in one pointing at the other. Then something will happen resulting in a related row being deleted. It’s probably deleted because I wanted it to be deleted. The thing is, Postgres doesn’t like this. It’s invalid data. You will now get all sorts of errors from Postgres because the data is invalid. This is going to cause all, or part, of your application to go down with errors.

MySQL ignores this, and wt ill continue servicing your queries. Your application will see no downtime. For normal people with normal sites, having their web site stay up without error is a hell of a lot more important than a dangling foreign key. In most real world cases, that broken foreign key constraint won’t cause any issue whatsoever. Pages not serving is a gigantic problem for even the smallest site.

Another problem Postgres used to have is lack of replication. Every place I have ever worked with MySQL has used the master/slave configuration. It’s just so easy to setup, and has been for a long time. Postgres just added the feature, so how much can you trust it? Even if you do trust it, it’s very hard to configure it properly. There are third party solutions like PL/Proxy and Slony, but they are even more difficult to configure. MySQL makes it easy, and that is my final point.

MySQL is very easy. It has extremely user friendly commands such as ‘show databases’ or ‘show full processlist’. In Postgres, you have to learn commands like ‘\l’ and ‘\d’. I’m a vim user, so I have no problem with this, but what if you’re some guy with a blog and not a vim using developer? Postgres is just really cryptic. MySQL is much more forgiving of the user, and more accessible to normal people.

Another example of MySQL’s ease of use is in its authentication model. Granted, it’s not all that simple. You specify user@hostname with a password. Then you give that user permissions on different databases, or global permissions. It’s a little complex, but it’s comprehensible to normal people. It’s the same concept as the permissions they setup when they do a PHPBB, or something like that. The administrative user has all global permissions. The moderator user can update this table, insert that table, but only select these other tables.

Postgres has the pg_hba.conf. The first time I encountered it, it took about 20 minutes of reading documentation before I understood what the hell was going on. Yes, I fully admit that this configuration gives a lot more power and a lot more options than the MySQL way of doing things. The thing is, none of those options are useful for every day people. The only thing I ever do in terms of database permissions is I create a new MySQL user and password with full permissions to each specific database. Then I give that username and password to the application in question. In MySQL, this is the default, and PHPMyAdmin makes it easy. In Postgres, that is about as far from the default as you can get.

There has been a migration of database popularity, especially in the web application world. People started with MySQL. They were upset about data integrity, so they moved to Postgres. Then they were upset about slowness, so they moved to “NoSQL.” The thing is NoSQL has all the same problems as MySQL. Have you not read all these stories about people losing all their data in MongoDB? Losing all your data is a much bigger problem than a dangling foreign key or a broken transaction.

Also, people seem to forget that MySQL has multiple choices of storage engine. Most of the complaints with MySQL data integrity are actually complaints with the MyISAM storage engine. If you switch MySQL to use the InnoDB storage engine, many, but not all, of those complaints are now invalid. The thing is, most people don’t need InnoDB, the same way they don’t need Postgres. I actually think that anyone using InnoDB should use Postgres instead. InnoDB is a lot slower than MyISAM.

Look at all the companies people are talking about that are supposedly using NoSQL. Digg, Twitter, Facebook, bit.ly. People say things like “bit.ly uses MongoDB.” What they don’t tell you is that bit.ly uses MongoDB for just one feature of the site. The actual core of the bit.ly service, the URL shortening, still uses MySQL. The same is true at Facebook. They use Cassandra for some stuff, but the core of Facebook is still MySQL. If it’s good enough for them, then it works. If it works, it’s good.

I understand there are some Postgres/database type people out there who are probably going insane if they read this. They have some sort of OCD-like situation where they need all the data to be exactly perfect, or it drives them crazy. You know, if I look at some of my MySQL database, there is bad data in there. I see it with my own eyeballs. It’s a tiny bit annoying. But then I look at my actual application. What do you know, my application is still running. It has no errors anywhere. It’s fast. It’s working. That’s all that matter. Everything else is just anal developers who need to get over it.

If you’re doing a serious big application, use Postgres, please. I do not want data errors ruining my vacation when I book a flight. I do not want my bank to lose my money. If you’re doing another every day web site, use Postgres if you want, but it’s not necessary. MySQL is faster, easier, and it works. The web site will load, and we can all rest more easily.

The New Google News

This guy Bruce Tognazzini (Tog), really doesn’t like the recent changes to Google news.

The team involved appears unwilling or unable to just come clean with the fact that they screwed up completely, and, instead, are just chipping away at one individual screw-up after the other. It’s as though they replaced their gazelle with a pig and are now carving away at the pig, desperate to make it at least look like a gazelle. So far the result, unfortunately, it does not look (or feel) like a gazelle at all, but only like a distressed and wounded pig.

Top 10 Reasons the New Google News Sucks

I read Google News a lot. I visit it multiple times a day. It is by far the best news aggregator in existence. It doesn’t show you any stupid crap, because, as with all Google systems, it’s algorithmic as opposed to democratic. It is incredibly intelligent when it comes to grouping duplicate content. It’s highly customizable, and it manages to cover everything important.

Recently, they changed Google News. They changed the layout from two columns to one, and some other things. As someone who visits that site frequently, on my PC and on my iPhone, I’m indifferent. It’s neither better nor worse than it was before. All of the key reasons to visit Google News, which I have already described, have not changed. It still provides links to all the important news stories without duplicates or stupid crap.

Remember the hundred or so times Facebook changed its layout and/or privacy policies and people were in an uproar? Well, everyone is still using Facebook, still playing Farmville. No changes going on there. No surprises.

One of my favorite stories is about the Citgo sign in Boston. It’s a gigantic and famous outdoor advertisement. It’s quite garish. When they originally wanted to install it, the neighborhood fought against it tooth and nail. They lost. More recently when it was suggested that it be taken down, people fought to keep it up because it had become a landmark.

Remember when they changed the name of the Nintendo Revolution to the Nintendo Wii? Remember people made jokes about the name? Remember when Firefox changed it’s name two or three times? Now, everyone calls them Wii and Firefox without even thinking about it.

Every time anything changes, for better or for worse, people complain. Then after a time, the change almost always sticks and wins and the complainers forget. We’ll never eliminate all this moronic griping, but perhaps we can lessen it slightly. The next time something changes, wait a week or two before you decide whether the change is really so bad. Odds are you will get used to it and forget it was ever different. The next time you see or hear someone complaining about a recent change, simply because it is a change, ignore them, or at least give them the same advice of waiting a week or two.

The more things change, the more they stay the same. Change is the only constant. You’ve heard it all before. That’s why we don’t want to hear it again. Save your breath for when there’s a really important change with real consequences. Losing your shit over the layout of a web site is kind of sad.

Techies care about education, but not enough

In a posting on LearnBoost Rafael claims that technologists care about education, but don’t care enough to actually do anything to solve the problems of education. He’s completely right, and he’s also completely wrong. He doesn’t even try to think of a single reason why this is the case. I will tell you those reasons.

This is why you see these passive signals of techies caring about education via the repeated upvotes on communities like Hacker News. Techies care a little bit, but still it’s not enough to drive change. Another way of putting it: there is a lot of demand for change in education but not enough people supplying that change. So we keep repeatedly celebrating the few education innovators.

Techies care about education, but not enough

Let me tell you something Mr. Rafael. I care a great deal about education. If I could pick any job in the world, I would pick to be a technology teacher in a high school somewhere. Now, why aren’t I currently employed in that dream job of mine? It’s not because I didn’t try. The reasons I am not a high school computer teacher are the same reasons that technologists don’t care about education “enough”.

Reason number one is money. Technology skills are very rare, and very valuable. Almost all of my college professors worked in the private sector during the summer because that’s where they made their real money. Even with all the extra benefits from the teacher’s unions, a teacher’s salary isn’t even remotely enough compared to what I can make at even a normal tech job. If you want technologists to help with education, you have to pay. The reality of the world is that corporations have a lot more money than schools, so that’s where technologists go. I don’t know how well a school would handle a brand new computer teacher being paid more than the principal.

If it’s just a matter of money, I look like a greedy tool, so let’s look at the other two issues. The first of which is lack of freedom for the teacher. Most teachers do not have freedom to teach as they please. They have to follow curriculum. They have to teach a certain way. No decent technologist can tolerate that. They would demand to teach the subject matter they want in the manner of their own choosing. A technologist would chafe if they were told what to do or how to do it.

Even worse, many schools will not hire you to just teach technology. I looked into it, and most schools will hire you has a math or science teacher with the opportunity to teach one or two technology courses, if that. That just isn’t going to cut it. I have no interest whatsoever in being a math or science teacher, grading papers, or other bullshit. I want to have a classroom full of computers, and other stuff, and I want to be the complete dictator of that room. The school exists only to put students into and out of that room.

That ties into the last problem which is the bureaucracy of the school as a whole. You see plenty of stories all over tech sites about a kid being suspended for having his boy scout knife in school, or some other nonsense. Back in my high school I got reprimanded for installing the flash plugin on a school computer. Technologists are nothing if not infuriated at the moronic policies and limitations that are placed on students and on the school computers. I imagine I, and other similarly minded folk, would be fired almost immediately by some cowardly principal who always follows policy to the letter.

If you want technologists to help with education, you have to pave a path for us. Because our technological knowledge is rare, we have great power. We are holding all the cards. Therefore, you have to grease the wheels if you want our help. Even if we want to help you, we can’t as long as the walls of bullshit are still in our way. Even if you can’t offer us as much money, you have to eliminate the bureaucracy and idiocy. We won’t tolerate moronic school policies whether they are applied to us or to students. We’ll probably show up at the school, tell everyone they are stupid, wave the middle finger around, and be sent home, if not to prison.

We would love to come and help you, but I think it will have to wait some time. Call us when you’re really ready for technological egos to roam your halls.

NoSQL or NoJoin?

Daniel Lamire has written an interesting post discussing the terminology we should use to refer to the increasingly popular non-relational databases such as BigTable, MongoDB, CouchDB, etc. He’s mostly discussing semantics, but it got my mind thinking on this issue just one more time.

However, in practice, SQL is based on joins and related low-level issues like foreign keys. SQL entices people to normalize their data. Normalization fragments databases into smaller tables which is great for data integrity and beneficial for some transactional systems. However, joins are expensive. Moreover, joins require strong consistency and fixed schemas.

In turn, avoiding join operations makes it possible to maintain flexible or informal schemas, and to scale horizontally. Thus, the NoSQL solutions should really be called NoJoin because they are mostly defined by avoidance of the join operation.

NoSQL or NoJoin?

Because these databases have become very popular recently, I have tried to learn a great deal about them. I even went to a conference about MongoDB. What I have learned is that these databases are very fast, and more easily scalable than relational databases. I have also learned that not having a strict schema for each table is advantageous in many applications. However, there is still this circle of logic I can’t get out of my mind.

You’re using MySQL with MyISAM. That’s an unreliable and flawed database engine. It has no data integrity or compliance. Ok, so you switch to InnoDB or Postgres. Now you have the data integrity, but everything is slower and more difficult to scale or replicate. The solution, therefore is noSQL/noJoin databases which have none of the data integrity.

Wait, what? Why would you choose the non-relational database over MySQL with MyISAM? They have the same data integrity and transactional issues. They are both fast. They both scale and replicate easily. The only major difference I can see is that MySQL has a strict schema and the noSQL databases have no joins.

I don’t know about other people, but every database-powered application I have ever written includes many joins. There is a great deal you can do without joins, but eventually there is a feature that requires them. Let’s say you have authors and articles in two separate tables. The author gets married, and you want to change their last name. With SQL you just update the row in the author table. Without it, you have to search for every blog post with that author, and change the name. Talk about a pain.

Now, there is an alternative to this methodology. You could have an author record and put the unique ID of the author into each blog post. So now what are you doing? You’re writing a join by hand in your application. You get benefits of using the NoSQL database, but the consequence is that you have to write more code. If using a relational database, you could do a lot of work with a single line of SQL. Instead, you write an entire algorithm in your application to do that same work as an SQL statement. It’s reinventing the wheel. You’re duplicating the work of the MySQL developers.

Now what toll does this take on performance and scalability? Yes, the NoSQL databases scale so well, and are so fast. That’s because the application servers are doing all the hard work. It’s shifting the load, not removing the load. On top of that, how good a coder are you really? I personally do not think that I can write a join algorithm that is better than what the MySQL developers can write. If I have a database without joins, and I have to write a join for my application, then how can it possibly be better than if I had just used a relational database? If it is somehow faster and better, that can only mean there is a serious problem in the join algorithm in the relational database that must be fixed. I doubt it. If indeed it is better, then why don’t they just add join commands to the noSQL databases?

If anything in my logic is wrong, please point it out to me. I’m not trying to flame, I’m trying to learn. It’s just that no matter how hard I try, I can’t comprehend the supposed hueg benefit of the non-SQL database. At best it frees you of the limits of a schema while moving some load from the database server to the application server. If I had an application that would heavily benefit from being schema-free, and there were few joins, it might be worth it. For every real world application I have actually written, it makes little sense.

Then again, every application I have ever written has always had more strain on the application server than the database server. I’m sure if I worked at Facebook or some other incredibly large place, the scaling benefits would emerge. It would be worth the effort of writing that extra code and moving some of the heat over to application servers. The thing is, if that’s my problem, I’m in a good spot. It means I’ve got so much demand that I’ve probably got money to pay someone else to do that work. There’s still little reason to start out in the beginning with anything but SQL. You’re likely making more work for yourself in exchange for scaling benefits that you do not need because you are small.

I’m sure there are some applications out there that have few, or even zero, joins. I’m also well aware of many applications where being free of a schema is such a huge boon, it is worth the price of writing more code. These applications exist, but in my experience they are rare. Almost every application I can conceive of is naturally relational in some way.

If anyone out there can explain what I am missing, I will be very thankful. Until then, I’ll be using MySQL for almost everything. It’s easy. It works. It’s fast. I don’t need the super data integrity, or user unfriendliness, of Postgres. And while it may not scale as easily as others, it scales much larger than any application I have ever had to deal with. If I am ever so fortunate to have to scale large, then I’ll make a move. Until then, I’ll save myself the trouble.

Not Implementing Features Is Hard

Robert O’Callahan, a Mozilla developer, has made a blog post explaining why Firefox omits certain features. Based on the headline, I thought this was going to be discussing the evils of feature creep. That is not the case.

Sometimes we take heat for deliberately not implementing certain Web platform features. Three significant examples are SVG Fonts, WebSQLDatabase and H.264 in <video>.

But I honestly think we’re doing the Web a service by resisting these things.I think not implementing certain features is one of the hardest and most altruistic things we do.

Update I should also point out that we’re not just exerting “stop energy” here; for each of these features that we’re not adding, we are promoting superior alternatives — namely WOFF, IndexedDB, and WebM.

Not Implementing Features Is Hard

I’ve been using Firefox since it was called Phoenix. In those days, your only real browser choices were Internet Explorer or the full Mozilla. Mozilla was a better browser, and even had a crude tabbed browsing implementation. The problem was that it was big and slow because it wasn’t just a browser. It also included email, newsgroups, composer, and more. Imagine if all the different parts of Microsoft Office were a single program. That was Mozilla in those days.

Over time Phoenix, and then Firefox, gained popularity for three reasons. It removed all those features that most people didn’t want or use. This resulted in vastly increased performance and stability that other browsers had a hard time competing with. It removed even more features that some people actually did use, but made them available by extensions. Lastly, they added and polished new features that most users actually were using, such as tabbed browsing.

There was one more thing Firefox did that was very important. Most web sites actually worked in it. This was a time when many sites were built only for Internet Explorer. People would begrudgingly continue to use IE simply because some web site required it. Because most sites worked in Firefox people were able to switch. This is important because it shows that people are more loyal to web sites than web browsers. People will switch to a better browser, but only if all of “their” sites work in it. This is why businesses with Internet Explorer specific intranet apps stick with it, even when the IT staff knows that better browsers exist. The web apps take precedence.

Today we have some browser features like SVG Fonts, WebSQLDatabase, and H.264 video. If Firefox wanted to make an argument that they shouldn’t included these features at all, because of feature creep, that’s actually cool. That’s the philosophy of Firefox. Unless it’s for every user, put it in an extension or a plugin. Keep the browser small.

Too bad that’s not what they’re doing here. They are supporting what they feel are “superior” alternatives. Those alternatives may indeed be superior for legal reasons, or even technological reasons, but that doesn’t mean jack or shit. Web sites are already using these features that Firefox is not including, and the more sites that don’t work in Firefox, the less people will use Firefox.

As a web developer, Firefox is the go to browser. I’m using it right now. Why do I use it? It’s because it has Firebug. I can’t do my job without Firebug. The webkit developer tools are getting a lot better. Most developers I know only use Firefox because there is some extension they need. They really want to stop using Firefox, but can’t. Chrome has extensions now, and most of the important ones are being ported. There’s less and less tying people to Firefox, and if apps work better in webkit browsers, people are going to switch away. It might take some time, but it will happen.

What bothered me most about this article was that he says that they’re doing the web a service. I can safely say, they are doing the web a huge disservice. Let me use something people haven’t paid much attention to as an example. Behold the HTML5 audio tag. It’s just like the video tag, but for audio. Can you guess which browsers support which audio codecs?

Codec support in modern browsers
Browser Ogg Vorbis MP3 WAV
FireFox 3.5
Safari 4
Chrome 3 (beta)
Opera 10 (beta)

Native Audio in the browser

This chart can easily be redone to show support for video codecs, fonts, etc. Just about any new HTML feature follows this same pattern. Yes, all browsers are equally to blame here. If Safari supported Ogg, we would be all set. If Chrome supported wav, that would also solve everything. It takes just one to break the dam.

That being said, I don’t know why none of them do it. The browser that supports the full set will be the browser that works with any web app. Thus, it will be the browser that dominates. Firefox could be that browser, they just have to give up on their philosophical bullshit. Web developers will have peace of mind. The web will become a much better place because there won’t be all sorts of hacks in web applications to switch between different codecs depending on user agent.

There is something to be said for leaving out features completely for the sake of performance and stability. The extension philosophy has got Firefox to the place it is today. Even so, making sure every web site works in your browser is priority number one. And sometimes adding a feature, like tabbed browsing, is beneficial to everybody. These things Firefox is holding out on should be included straight up. I’m starting to think maybe it would be a good idea for someone to create a Firefox fork that has all these features that Mozilla is refusing.

Well, it’s no skin off my back. If they continue to hold out, Chrome and others will just eat their lunch. They say they don’t care about market share, so that’s totally ok by them. It makes me wonder. If they don’t care about market share, why did they always so proudly advertise their download figures?

Is Google far too much in love with engineering?

Chris over at Cnet discusses the fact that almost every single employee at Google is an engineer, and whether or not that is entirely a good thing.

It’s nice if a pilot has a background in flying. It’s really quite special if a colonoscopist has a background in medicine. But does everyone who heads up a department at Google really need to have a background in engineering?

Is Google far too much in love with engineering?

If you’re not aware of the phenomenon of reactionary extremism, you obviously don’t read anything on the web. When people see a problem with something, their brains somehow create a false dichotomy. There’s a problem with copyright law, therefore all intellectual property law is bad. Capitalism is causing some badness here, therefore socialism is perfect and the best!

Google has almost definitely fallen victim to this fallacy. I know that almost every developer has experienced the pain of having to obey poor decisions from managers who do not understand technology. They make bad decisions based entirely on money or gut feelings, or other factors, and it infuriates the engineer. Google was started by engineers, so this situation angers them more than anything else. Their highest priority is to make sure it never happens.

Is this really a problem? Not if you’re a Google employee. If you’re a Google employee, you’re likely an engineer. Therefore you also know the frustration of managers who know nothing about technology poor decisions making engineers miserable. You are glad that you are shielded from them. You are glad that even if a decision doesn’t go your way, it was because someone had numbers, and there is some evidence behind the decision.

The problem is that by having only engineers, Google has some extreme weaknesses. There are two major ones that I think stand out above all others.

The first is user interface design. Sure, Google does a lot better than many, but not good enough by far. Android and Google Wave have some of the worst user interfaces imaginable. Google apps are sort of ok-ish. Chrome is alright. Nothing amazing. They have effectively unlimited money. They can design a new datacenter out of shipping containers that is crazily energy efficient. Why can’t they hire a decent UI person? Oh, wait, they just did. We’ll see if they actually listen to him, or if engineers decisions override him. Even so, it’s just one person, and it’s way too late in the game.

The second major problem is that they don’t finish or integrate any of their products. Google Voice is the greatest example of this. Once a person comprehends the idea of Google Voice, they immediately and desperately want it. The problem is that you can’t transfer your old number to it. It also doesn’t “just work.” Even on Android phones with Google’s logo printed on the phone itself, Google Voice does not integrate seamlessly and perfectly 100% of the time. They have so much money, and Google Voice has been out for so long. They should be able to put the polish on it.

I know, as a developer, that the old saying about the final 10% of the work taking 90% of the time is absolutely true. The last part is always the hard part. Google is full of engineers, so they know that rule as well as anybody. It seems to me that they all know this, and they all collude so that nobody ever has to actually do that hard work which is so unpleasant for engineers. They just keep doing the 90% of the work that takes 10% of the time. It’s truly an engineer’s paradise.

The result is that Google has a ton of projects. There are hundreds and hundreds of Google things that they have created or purchased. Which ones have actually made it big besides search, ads, mail, chat, analytics, docs. Other than search and ads, even things like docs is a drop in the bucket in terms of popularity. YouTube was popular before they bought it, so that doesn’t count. They have lots of projects, and almost no completed ones. It’s because they’re all engineers doing what engineers want to do. They need a non-engineering business person to crack the whip and make them do the work they don’t want to do. The work they don’t want to do is the work that will actually put them ahead of the competition.

Even if they realize this, there is another problem. They don’t need to care. They make so much freaking money just from search and ads, why do they care if a project like Voice is half-baked? It only hurts users. Users aren’t engineers who work at Google, so who cares if they suffer? They don’t even need to give them the courtesy of a phone number to call for support, because those non-engineers can’t possibly have anything valuable to say.

Google, hire some UI people, and do what they say, even if you don’t like what they have to say. They know better than you. Secondly, stop making new stuff. Finish the things you have. You might not think it’s worth it to invest more work into something that isn’t a hit. Well, nothing you make is going to be a hit unless you’ve worked out every last kink.

Obviously problems arise when non-engineers make decisions related to engineering. Google has clearly solved that problem, and it gives them an advantage. However, when you have engineers making decisions about law, art, or anything else that isn’t engineering, the same principle applies. Perhaps it is just the anti-social nature of engineers that prevents Googlers from working well with other disciplines. They better make some friends, and soon.