MySQL is Great, Haters Gonna Hate

Posted on July 24, 2010 by apreche

I made a mistake that I must correct. I stupidly broke my rule of only writing comments here, and made a couple comments on this post complaining about a bunch of problems in MySQL.

After being a happy PostgreSQL user for years, I’ve finally had to use MySQL for the last 6+ months.

I have to admit, I was starting to think it wasn’t all that bad–that the “MySQL is a toy” line was outdated.

It’s not.

– Dammit, MySQL

I’ve been developing database applications, mostly for the web, for seven years now, professionally for five. I’ve used every kind of database for every kind of app. I’ve used MySQL, Postgres, sqlite, CouchDB, and some others. I’ve also worked on applications ranging in size from a tiny blog nobody reads to gigantic high traffic applications with millions of rows, or millions of hits.

Lately I have noticed that there is a very strong backlash against MySQL. That original blog post was linked from Hacker News, and if you look at the comments there, everyone who is in favor of MySQL is getting downvoted.

Let me tell you something. All these technological complaints people have with MySQL, they are 100% true. MySQL has no transactions. Its foreign key constraints are not up to par. It doesn’t follow standards perfectly. There are very real possibilities of your data becoming invalid, or even corrupt. MySQL is far from being a super perfect database under the hood.

If all those things are true, then MySQL sucks, and we should never use it, right? Wrong. Even though MySQL has these issues, they simply do not matter for the overwhelming majority of applications. If you’re working for a bank, a hospital, or the military, then MySQL is right out. In those cases, reliability and data integrity are so important, and the system is so large, you just can’t be using MySQL. I’m willing to bet you’re not doing any of those things, are you? You’re doing a blog, or maybe a social network.

You might say that you are doing a blog, but PostgreSQL and MySQL are both free and open source. You may not absolutely need the data integrity and standards compliance, but why not have it anyway? It won’t hurt, right?

I disagree with this. In my many years of using both Postgres and MySQL, my experience is that Postgres is extremely finnicky. Because it demands standards compliance and data integrity, it demands a lot more work from the application developer. Let me give some examples I have encountered in the real world.

Transactions are a big reason Postgres is so great. You can make sure your data is always correct because a group of writes will go in all together. This is very important in a serious application. The thing is, if you don’t need transactions, using them can cause you a great deal of trouble. In building multiple Django applications on Postgres, I have seen a very frequent error when a transaction has been opened, but not closed. The result is that the entire database is stuck. You can’t read or write from it until the transaction is closed out. Whenever this happens, I just restart the db server, and everything is back to normal.

It shouldn’t be possible for this to happen! The fact that restarting the database fixes it is proof that it doesn’t even need to happen. Yes, transactions are good, but a small user error causing the entire database to go out of service is much worse. In a huge application, you can take the time to test and make sure you don’t make those errors. For a normal every day application, you just want it to stay up. You don’t care about some unclosed transaction. Just let it slide.

A similar situation happens with the much lauded foreign key constraints. I’ll have myself two tables with a foreign key in one pointing at the other. Then something will happen resulting in a related row being deleted. It’s probably deleted because I wanted it to be deleted. The thing is, Postgres doesn’t like this. It’s invalid data. You will now get all sorts of errors from Postgres because the data is invalid. This is going to cause all, or part, of your application to go down with errors.

MySQL ignores this, and wt ill continue servicing your queries. Your application will see no downtime. For normal people with normal sites, having their web site stay up without error is a hell of a lot more important than a dangling foreign key. In most real world cases, that broken foreign key constraint won’t cause any issue whatsoever. Pages not serving is a gigantic problem for even the smallest site.

Another problem Postgres used to have is lack of replication. Every place I have ever worked with MySQL has used the master/slave configuration. It’s just so easy to setup, and has been for a long time. Postgres just added the feature, so how much can you trust it? Even if you do trust it, it’s very hard to configure it properly. There are third party solutions like PL/Proxy and Slony, but they are even more difficult to configure. MySQL makes it easy, and that is my final point.

MySQL is very easy. It has extremely user friendly commands such as ‘show databases’ or ‘show full processlist’. In Postgres, you have to learn commands like ‘\l’ and ‘\d’. I’m a vim user, so I have no problem with this, but what if you’re some guy with a blog and not a vim using developer? Postgres is just really cryptic. MySQL is much more forgiving of the user, and more accessible to normal people.

Another example of MySQL’s ease of use is in its authentication model. Granted, it’s not all that simple. You specify [email protected] with a password. Then you give that user permissions on different databases, or global permissions. It’s a little complex, but it’s comprehensible to normal people. It’s the same concept as the permissions they setup when they do a PHPBB, or something like that. The administrative user has all global permissions. The moderator user can update this table, insert that table, but only select these other tables.

Postgres has the pg_hba.conf. The first time I encountered it, it took about 20 minutes of reading documentation before I understood what the hell was going on. Yes, I fully admit that this configuration gives a lot more power and a lot more options than the MySQL way of doing things. The thing is, none of those options are useful for every day people. The only thing I ever do in terms of database permissions is I create a new MySQL user and password with full permissions to each specific database. Then I give that username and password to the application in question. In MySQL, this is the default, and PHPMyAdmin makes it easy. In Postgres, that is about as far from the default as you can get.

There has been a migration of database popularity, especially in the web application world. People started with MySQL. They were upset about data integrity, so they moved to Postgres. Then they were upset about slowness, so they moved to “NoSQL.” The thing is NoSQL has all the same problems as MySQL. Have you not read all these stories about people losing all their data in MongoDB? Losing all your data is a much bigger problem than a dangling foreign key or a broken transaction.

Also, people seem to forget that MySQL has multiple choices of storage engine. Most of the complaints with MySQL data integrity are actually complaints with the MyISAM storage engine. If you switch MySQL to use the InnoDB storage engine, many, but not all, of those complaints are now invalid. The thing is, most people don’t need InnoDB, the same way they don’t need Postgres. I actually think that anyone using InnoDB should use Postgres instead. InnoDB is a lot slower than MyISAM.

Look at all the companies people are talking about that are supposedly using NoSQL. Digg, Twitter, Facebook, bit.ly. People say things like “bit.ly uses MongoDB.” What they don’t tell you is that bit.ly uses MongoDB for just one feature of the site. The actual core of the bit.ly service, the URL shortening, still uses MySQL. The same is true at Facebook. They use Cassandra for some stuff, but the core of Facebook is still MySQL. If it’s good enough for them, then it works. If it works, it’s good.

I understand there are some Postgres/database type people out there who are probably going insane if they read this. They have some sort of OCD-like situation where they need all the data to be exactly perfect, or it drives them crazy. You know, if I look at some of my MySQL database, there is bad data in there. I see it with my own eyeballs. It’s a tiny bit annoying. But then I look at my actual application. What do you know, my application is still running. It has no errors anywhere. It’s fast. It’s working. That’s all that matter. Everything else is just anal developers who need to get over it.

If you’re doing a serious big application, use Postgres, please. I do not want data errors ruining my vacation when I book a flight. I do not want my bank to lose my money. If you’re doing another every day web site, use Postgres if you want, but it’s not necessary. MySQL is faster, easier, and it works. The web site will load, and we can all rest more easily.

Posted in Technology | Leave a comment

Raw food raids are on the rise

Posted on July 20, 2010 by apreche

There’s an interesting post on BoingBoing about government raids on farms that are selling wares at raw food markets. It’s simultaneously ridiculous and also awesome.

“If you take my computer again, I can’t do my homework.”—Words spoken to agents raiding a small California dairy farm by the owner’s 12-year-old daughter. The farm is known for producing raw goats’ milk cheese that is sold by members-only raw food markets. Grist reports that local, state, and federal raids on alternative “raw foodie” membership marketplaces are on the rise.

– Raw food raids are on the rise

Ok, so the ridiculous part of this is the computer confiscation. People need computers and Internet access. With news, banking, and even government all being online, someone without a computer and Internet is effectively cut off. Really, we need to follow the lead of Finland and make Internet access a right that can not be revoked unless someone is convicted of a crime of appropriate severity.

As for the issue of the food, hells yes. If you read the posts on BoingBoing, they’re typical crazy hippies who think that anything which happens to benefit a bigger company to the detriment of a smaller one is an evil corporate conspiracy. Those same people will cry huge foul when they find a big company doing something like selling toys with lead paint. They’re just hypocrites who pick sides based on which party they empathize with more, rather than based on the facts of the issue itself.

The facts are that raw milk, raw cheese, and other such foods are fucking poison. You know humanity is only where it is today because we invented alcoholic beverages. Why is that? It’s because alcoholic beverages didn’t turn to poison and kill us all. The alcohol in the beverage killed off the nasty bacterias.

If you drink some raw milk or some bathtub cheese from one of these hippy farms, your odds of getting sick are quite high indeed. You might say that it’s never happened to you, and that’s quite possible. But it’s a huge risk. You want to get listeria, seriously?

You know how come we drink milk today? Pasteurization. Yeah, you know, the process that pretty much saved all of our collective asses. Without pasteurization, we wouldn’t be able to eat half the shit we eat today without getting sick all over the place. I’m not just talking about the common case with lots of diarrhea and vomiting, death is a significant possibility here.

You might like the taste of raw milk, but I’ve heard that anti-freeze tastes really good. I guess anti-freeze is more dangerous than raw milk, as it almost guarantees death, but the milk isn’t far behind. Actually, I’d say that lead paint is probably less dangerous than the milk. I mean, you aren’t supposed to eat the paint, and it can’t really hurt you if you leave it on the wall where it belongs. You’re going to drink the milk, and if it’s unpasteurized, you know what’s living in there?

Remember, milk comes from a farm. Farms are dirty places. They are really dirty. Even a clean farm is dirty. There’s animal poop and dirt and other nasty stuff all over. There’s also all kinds of bacteria that would love to fuck you up. The evil commercial farms are actually heavily regulated. They go to great lengths to make sure the food is clean. Ultra pasteurization, irradiation, and all kinds of other technology that make food safe to eat, and help fight hunger. Science scares hippies, so they don’t like this sort of thing, even though it saves lives. The hippy farms, they hardly do anything. Remember, unpasteurized milk is milk with cow poop in it. The bacteria in that cow poop can kill you.

The reason we have these laws for consumer protection is because clearly consumer education doesn’t work. There’s no way we can teach everyone the dangers of raw food. And even if we do, people like these hippies on BoingBoing still won’t believe it. Idealistically I am anti-nanny state. Pragmatically, let’s save some people’s lives. Even if they’re stupid enough to drink raw milk, they don’t deserve to die or have their insides come out both ends for that stupidity. Saving them from themselves is a good move. Government, please crack down more on unclean farms.

Posted in Society | Leave a comment

The New Google News

Posted on July 19, 2010 by apreche

This guy Bruce Tognazzini (Tog), really doesn’t like the recent changes to Google news.

The team involved appears unwilling or unable to just come clean with the fact that they screwed up completely, and, instead, are just chipping away at one individual screw-up after the other. It’s as though they replaced their gazelle with a pig and are now carving away at the pig, desperate to make it at least look like a gazelle. So far the result, unfortunately, it does not look (or feel) like a gazelle at all, but only like a distressed and wounded pig.

– Top 10 Reasons the New Google News Sucks

I read Google News a lot. I visit it multiple times a day. It is by far the best news aggregator in existence. It doesn’t show you any stupid crap, because, as with all Google systems, it’s algorithmic as opposed to democratic. It is incredibly intelligent when it comes to grouping duplicate content. It’s highly customizable, and it manages to cover everything important.

Recently, they changed Google News. They changed the layout from two columns to one, and some other things. As someone who visits that site frequently, on my PC and on my iPhone, I’m indifferent. It’s neither better nor worse than it was before. All of the key reasons to visit Google News, which I have already described, have not changed. It still provides links to all the important news stories without duplicates or stupid crap.

Remember the hundred or so times Facebook changed its layout and/or privacy policies and people were in an uproar? Well, everyone is still using Facebook, still playing Farmville. No changes going on there. No surprises.

One of my favorite stories is about the Citgo sign in Boston. It’s a gigantic and famous outdoor advertisement. It’s quite garish. When they originally wanted to install it, the neighborhood fought against it tooth and nail. They lost. More recently when it was suggested that it be taken down, people fought to keep it up because it had become a landmark.

Remember when they changed the name of the Nintendo Revolution to the Nintendo Wii? Remember people made jokes about the name? Remember when Firefox changed it’s name two or three times? Now, everyone calls them Wii and Firefox without even thinking about it.

Every time anything changes, for better or for worse, people complain. Then after a time, the change almost always sticks and wins and the complainers forget. We’ll never eliminate all this moronic griping, but perhaps we can lessen it slightly. The next time something changes, wait a week or two before you decide whether the change is really so bad. Odds are you will get used to it and forget it was ever different. The next time you see or hear someone complaining about a recent change, simply because it is a change, ignore them, or at least give them the same advice of waiting a week or two.

The more things change, the more they stay the same. Change is the only constant. You’ve heard it all before. That’s why we don’t want to hear it again. Save your breath for when there’s a really important change with real consequences. Losing your shit over the layout of a web site is kind of sad.

Posted in Society, Technology | Leave a comment

Sell

Posted on July 7, 2010 by apreche

Douglas Rushkoff thinks that shit is about to go down, economically speaking.

Yes, this is really it. The beginning of a true end-of-cycle economically.

If you own “stocks,” use these bounces to get out completely. If you have to park your money somewhere, consider yourself lucky you have money to park.

– Sell

I’m not going to discuss whether he’s right or wrong. Time will tell, also I have no idea. I would however, like to add some counter advice. Well, maybe it’s not so much counter advice as it is additional advice.

Let’s pretend that he’s right and the market is about to go down, as in way down. If you own a lot of stock right now then yes, selling would be a good idea.

The rest of his advice, about creating actual value, is good advice in any economic scenario, and I wholeheartedly support it. However, selling now doesn’t mean avoiding stocks forever.

If the markets crash hard, I personally will likely buy like crazy. I’m young. I have a lot of time left. I have money. I have a 401k. If it goes way down, I’m going to increase my 401k contribution to the max I can afford. I might even just get an online trading account and just go for it.

See, I’m in it for the long haul. I’m not even 30 yet. Any investment I make will probably last 40 years plus. Buy low, sell high. That’s the most basic advice in all trading. There’s almost no chance that it will be lower when I retire than it will be if there’s a big dip soon.

It’s almost like a gift. Hey, maybe you didn’t start your 401k as early as you should have. Maybe you didn’t make your contribution large enough. If you have money afterwords, it’s your chance to play catchup.

If you believe Doug, then by all means sell now. But if he’s right, be prepared to buy after the smoke clears.

Posted in Society | Tagged | Leave a comment

Almost All Conventions Straight-Up Lie About Attendance

Posted on July 7, 2010 by apreche

My favorite convention in the entire world, PAX, has committed to Boston for three more years. I have attended the past three PAXes, and have spoken at all three. I plan to continue to do so for the foreseeable future. They have apparently just signed a deal to keep PAX East in Boston for three more years.

The Massachusetts Convention Center Authority and Reed Exhibitions, event organizer for the video gaming convention Penny Arcade Expo (PAX), signed an agreement recently that will bring the Seattle-based show back to the East Coast for at least the next three years.

In its first year at the Hynes Convention Center in March, PAX East drew nearly 60,000 gaming players and developers over the course of the sold-out three-day event.

– Video gaming conference PAX East commits to hub for three years

The only problem I have with this at all is the single lie. The lie is the sentence that says “PAX East drew nearly 60,000 gaming players and developers….” This is simply not the case. If someone goes to PAX East for the entire three day weekend, they are counted three times. I do not know their exact counting methodology, but they may also be counting staff, exhibitors, guests, and others in addition to the paying attendees.

I don’t mean to pick on PAX, my favorite convention of all. I go to a lot of conventions, and almost all of them tell the exact same lie. San Diego Comic Con, CES, New York Comic-Con, New York Anime Festival, E3, and more. All of these major conventions count attendance by turnstile. It throws the numbers off even more when you have a four day convention like SDCC, excuse me, Comic-Con International.

That being said, there are many conventions out there that do not lie at all. Otakon is a convention to note in this regard. All of the Otakon event stats are publicly available. I know for a fact that the attendance number they give is exact. It counts each human body exactly once. It does not count staff, exhibitors, etc. It only counts legitimate attendees once.

Perhaps, though, if the big conventions refuse to stop this farce, conventions like Otakon should begin to lie. If Otakon were to count turnstile, they could boast nearly 100,000 attendees, completely dwarfing even a big con like PAX. I have been to both Otakon and PAX several times. Without a doubt, Otakon is much larger in terms of physical space and population. There is no comparison when it comes to size. Comparing other factors is another matter entirely.

The thing is, I’m not even really picky here. If these articles about attendance would change their wording, I would have no cause to complain. For example, this article says “…60,000 players and developers.” That is a lie. They could have said something like “…an attendance figure of nearly 60,000.” That would be a true statement using the same number.

Imagine if you had a baseball stadium that seated 30,000 people. You sold it out every game. Assuming half of the 162 games are played at home, that’s 81 games. That’s 2,430,000 tickets sold. That’s well and good. However, if the stadium said that they “welcomed over 2,430,000 people this year”, that would be false. It is obvious that most of the tickets are held by season ticket holders. It’s the same butt in the seat every night. Assuming half the seats are season tickets, and the rest of the seats are never return visitors, that’s 1,230,000 unique people. These aren’t slight discrepancies we’re talking about here. These are numbers that are way off.

Now, you might ask what harm this lie does. Well, it is a harm to the conventions that do not lie. PAX gets lots of attention, far more than Otakon, and for many reasons. But perhaps if advertisers and the industry knew the true size of Otakon, they would pay it more attention. Perhaps some fans out there are deciding which convention to go to. Maybe they prefer big, or maybe they prefer small. Either way, they will make the wrong decision based on false information. Perhaps a business is deciding which convention to exhibit at. They will spend their money inefficiently at a smaller convention which lies, when they could perhaps get more bang for the buck at a truthful convention which is in fact larger.

We need to have some standard way of measuring event attendance. If we allow events to continue deciding their own attendance numbers based on whatever metric they decide upon, there will be no objectivity and no equal point of comparison. At the very least, the events should publish their attendance metric along with their figures to make it clear exactly how they are counting. Ideally, just cut the crap. Tell us the number of badges sold. As Otakon has shown, you know that exact number with an incredibly small margin of error, perhaps zero. If you want to brag about attendance, at least brag the truth. The true figures are still quite impressive, just let us compare apples to apples.

Posted in Conventions | 3 Comments

American Dream Is Elusive for New Generation

Posted on July 7, 2010 by apreche

I don’t even know where to begin with this article from the New York Times. It tells the “sad” story of a college graduate who has to live with his parents because he can’t get a job. Except that he did get a job, and he turned it down.

The daily routine seldom varied. Mr. Nicholson, 24, a graduate of Colgate University, winner of a dean’s award for academic excellence, spent his mornings searching corporate Web sites for suitable job openings. When he found one, he mailed off a résumé and cover letter — four or five a week, week after week.

Over the last five months, only one job materialized. After several interviews, the Hanover Insurance Group in nearby Worcester offered to hire him as an associate claims adjuster, at $40,000 a year. But even before the formal offer, Mr. Nicholson had decided not to take the job.

– American Dream Is Elusive for New Generation

There are so many things wrong here. Let me start with something simple. Why is he sending out four or five resumes a week? When I am on a job hunt, I can send out as many as five an hour. It’s clear from this fact alone that this kid is lazy and inefficient.

Well, there is one other explanation for how few applications he is sending. It could be that he’s just very picky. He claims that he’s willing to work anywhere, but apparently he’s not willing to do any job. He has some sense of entitlement that because he went to college he should immediately start in a high up position. Welcome to reality! He’s probably could apply to hundreds of jobs a week, he just doesn’t want those jobs because he thinks he’s too good for them. Guess what? Someone who will prefer to stay at home than to make $40k is an asshat. I wouldn’t hire him to flip burgers if I owned a McDonald’s.

The author of the article, Louis Uchitelle, places all the blame squarely on the economy. Oh, it’s so hard for these kids to find jobs. I don’t know what Louis is talking about, this kid found a job and turned it down. Blame the kid for being a douche, not the economy.

I graduated from high school in the year 2000, so I did not enter the job market until after the .com bust. I took the first job I could find. It paid $40k. I was working within a few weeks, and moved out soon after. Twice since then I have changed jobs. Each time I started my job hunt I have found new jobs in a matter of weeks for significant increases in pay. Even in this horrible economy, jobs are aplenty in my field.

Now, I don’t want to blame the kid entirely. I also want to blame his parents, and our whole society. My generation grew up with everyone telling us we should follow our passion and be whatever we wanted to be. They filled our heads with lies. This kid obviously still believes those lies. Not everyone can live their dream, most won’t. If we all lived our dream we would have too many astronauts and not enough farmers.

They also told my generation the lie that everyone should go to college, and that it’s a magical job ticket. Back in my parent’s generation going to college was special. Not everyone did it, and not everyone graduated. Thus, if you did have a degree, it was actually sort of a magical job ticket. Nowadays almost everyone has a degree, so the value of said degree in the job market is significantly reduced.

Because of these lies, my peers not only went to college, they majored in whatever they felt like. Many of them majored in things that provide absolutely no marketable job skills. I knew a person with a communications degree working at Gamestop, so awesome. Could have saved the money spent on college and gone to Gamestop right after high school and been a pretty big manager with four years of climbing the ladder. Four years of on the job experience is worth a lot more than four years of school.

Even worse than that, a lot of kids have gone to college with no direction. They went to college because it’s the thing to do, and their parent’s paid for it. They have no idea what they want to major in, or what they want to do with their lives. They toil in useless liberal arts classes doing nothing worthwhile, and perhaps never graduating. Even if they pick a major, they change their minds wasting thousands of dollars and years of their lives. The college doesn’t care, they’ll take your money.

Now, I did go to college. I went to RIT and majored in computer science. That happened to be both my passion and a marketable skill. It was easy to see from day one who was going to succeed and who would fail. There were kids who went home to their parents every weekend to have them do their laundry and feed them. There were kids in CS who had hopes of money, but no skill or interest. There were those who could work very hard and make the grade, but didn’t truly care about what they were doing.

The retention rates were horrible. A frighteningly large number of kids changed majors, transferred to other schools, or dropped out. They just couldn’t hack it. When actually forced to live independently and work hard, they quit. They boomeranged right back to mommy and daddy. It’s good, in a way, that we don’t have more job openings for these kids, as they would likely destroy whatever companies were fooled into hiring them.

I remember years ago I read a magazine that was left in a bathroom. There was an article in it about a couple who couldn’t get by because they were burdened with debt from college. At the time, their income was slightly less than mine, and their student loan debt was roughly equal. What was the problem? They had huge credit card debt. They had a gigantic house, kids, cars, luxuries. They lived as if they had a six figure income when they were in the lower five digit range. Yet the article blamed the economy rather than these people. They could have made a case using a better example, why did they choose these morons who were living so far above their means?

I think our friend in this article is quite a similar to that couple. He currently lives with his six figure parents. If he takes that $40k job, then his luxury level will decrease significantly. He won’t eat nearly as well. He won’t have as nice a television. He won’t have as nice a car. He won’t have an XBox. He refuses to step down before he takes a step up. Previous generations were forced to step down, by being drafted into war. The current one would rather leech off their parents.

And who can blame them? The parents must take a huge share of the blame. Why do they allow their kids to leech? They’ve spoiled these children their entire lives, why would they stop now? They’ve never forced, or even permitted, these children to live independently. They never will, unless they are tossed out in the street.

It’s well and good to have these kinds of stories out there. Tell the stories of the unemployed. Tell the story of any interesting person you can find. Just don’t go blaming the economy when the person is clearly at fault for their own problems. Find a better example. Find someone who has a marketable skill, is applying to jobs like crazy, and hasn’t even been called back. I think maybe the reason they don’t have such an example is because it doesn’t exist. Those people already have jobs, and the rest are not worth hiring. There isn’t a shortage of positions to be filled, there’s a shortage of quality labor.

I leave you with this relevant TED Talk by Mike Rowe. Make sure you watch the whole thing, or you won’t get it.

Posted in Society | Leave a comment

Techies care about education, but not enough

Posted on June 30, 2010 by apreche

In a posting on LearnBoost Rafael claims that technologists care about education, but don’t care enough to actually do anything to solve the problems of education. He’s completely right, and he’s also completely wrong. He doesn’t even try to think of a single reason why this is the case. I will tell you those reasons.

This is why you see these passive signals of techies caring about education via the repeated upvotes on communities like Hacker News. Techies care a little bit, but still it’s not enough to drive change. Another way of putting it: there is a lot of demand for change in education but not enough people supplying that change. So we keep repeatedly celebrating the few education innovators.

– Techies care about education, but not enough

Let me tell you something Mr. Rafael. I care a great deal about education. If I could pick any job in the world, I would pick to be a technology teacher in a high school somewhere. Now, why aren’t I currently employed in that dream job of mine? It’s not because I didn’t try. The reasons I am not a high school computer teacher are the same reasons that technologists don’t care about education “enough”.

Reason number one is money. Technology skills are very rare, and very valuable. Almost all of my college professors worked in the private sector during the summer because that’s where they made their real money. Even with all the extra benefits from the teacher’s unions, a teacher’s salary isn’t even remotely enough compared to what I can make at even a normal tech job. If you want technologists to help with education, you have to pay. The reality of the world is that corporations have a lot more money than schools, so that’s where technologists go. I don’t know how well a school would handle a brand new computer teacher being paid more than the principal.

If it’s just a matter of money, I look like a greedy tool, so let’s look at the other two issues. The first of which is lack of freedom for the teacher. Most teachers do not have freedom to teach as they please. They have to follow curriculum. They have to teach a certain way. No decent technologist can tolerate that. They would demand to teach the subject matter they want in the manner of their own choosing. A technologist would chafe if they were told what to do or how to do it.

Even worse, many schools will not hire you to just teach technology. I looked into it, and most schools will hire you has a math or science teacher with the opportunity to teach one or two technology courses, if that. That just isn’t going to cut it. I have no interest whatsoever in being a math or science teacher, grading papers, or other bullshit. I want to have a classroom full of computers, and other stuff, and I want to be the complete dictator of that room. The school exists only to put students into and out of that room.

That ties into the last problem which is the bureaucracy of the school as a whole. You see plenty of stories all over tech sites about a kid being suspended for having his boy scout knife in school, or some other nonsense. Back in my high school I got reprimanded for installing the flash plugin on a school computer. Technologists are nothing if not infuriated at the moronic policies and limitations that are placed on students and on the school computers. I imagine I, and other similarly minded folk, would be fired almost immediately by some cowardly principal who always follows policy to the letter.

If you want technologists to help with education, you have to pave a path for us. Because our technological knowledge is rare, we have great power. We are holding all the cards. Therefore, you have to grease the wheels if you want our help. Even if we want to help you, we can’t as long as the walls of bullshit are still in our way. Even if you can’t offer us as much money, you have to eliminate the bureaucracy and idiocy. We won’t tolerate moronic school policies whether they are applied to us or to students. We’ll probably show up at the school, tell everyone they are stupid, wave the middle finger around, and be sent home, if not to prison.

We would love to come and help you, but I think it will have to wait some time. Call us when you’re really ready for technological egos to roam your halls.

Posted in Society, Technology | Tagged | 1 Comment

NoSQL or NoJoin?

Posted on June 28, 2010 by apreche

Daniel Lamire has written an interesting post discussing the terminology we should use to refer to the increasingly popular non-relational databases such as BigTable, MongoDB, CouchDB, etc. He’s mostly discussing semantics, but it got my mind thinking on this issue just one more time.

However, in practice, SQL is based on joins and related low-level issues like foreign keys. SQL entices people to normalize their data. Normalization fragments databases into smaller tables which is great for data integrity and beneficial for some transactional systems. However, joins are expensive. Moreover, joins require strong consistency and fixed schemas.

In turn, avoiding join operations makes it possible to maintain flexible or informal schemas, and to scale horizontally. Thus, the NoSQL solutions should really be called NoJoin because they are mostly defined by avoidance of the join operation.

– NoSQL or NoJoin?

Because these databases have become very popular recently, I have tried to learn a great deal about them. I even went to a conference about MongoDB. What I have learned is that these databases are very fast, and more easily scalable than relational databases. I have also learned that not having a strict schema for each table is advantageous in many applications. However, there is still this circle of logic I can’t get out of my mind.

You’re using MySQL with MyISAM. That’s an unreliable and flawed database engine. It has no data integrity or compliance. Ok, so you switch to InnoDB or Postgres. Now you have the data integrity, but everything is slower and more difficult to scale or replicate. The solution, therefore is noSQL/noJoin databases which have none of the data integrity.

Wait, what? Why would you choose the non-relational database over MySQL with MyISAM? They have the same data integrity and transactional issues. They are both fast. They both scale and replicate easily. The only major difference I can see is that MySQL has a strict schema and the noSQL databases have no joins.

I don’t know about other people, but every database-powered application I have ever written includes many joins. There is a great deal you can do without joins, but eventually there is a feature that requires them. Let’s say you have authors and articles in two separate tables. The author gets married, and you want to change their last name. With SQL you just update the row in the author table. Without it, you have to search for every blog post with that author, and change the name. Talk about a pain.

Now, there is an alternative to this methodology. You could have an author record and put the unique ID of the author into each blog post. So now what are you doing? You’re writing a join by hand in your application. You get benefits of using the NoSQL database, but the consequence is that you have to write more code. If using a relational database, you could do a lot of work with a single line of SQL. Instead, you write an entire algorithm in your application to do that same work as an SQL statement. It’s reinventing the wheel. You’re duplicating the work of the MySQL developers.

Now what toll does this take on performance and scalability? Yes, the NoSQL databases scale so well, and are so fast. That’s because the application servers are doing all the hard work. It’s shifting the load, not removing the load. On top of that, how good a coder are you really? I personally do not think that I can write a join algorithm that is better than what the MySQL developers can write. If I have a database without joins, and I have to write a join for my application, then how can it possibly be better than if I had just used a relational database? If it is somehow faster and better, that can only mean there is a serious problem in the join algorithm in the relational database that must be fixed. I doubt it. If indeed it is better, then why don’t they just add join commands to the noSQL databases?

If anything in my logic is wrong, please point it out to me. I’m not trying to flame, I’m trying to learn. It’s just that no matter how hard I try, I can’t comprehend the supposed hueg benefit of the non-SQL database. At best it frees you of the limits of a schema while moving some load from the database server to the application server. If I had an application that would heavily benefit from being schema-free, and there were few joins, it might be worth it. For every real world application I have actually written, it makes little sense.

Then again, every application I have ever written has always had more strain on the application server than the database server. I’m sure if I worked at Facebook or some other incredibly large place, the scaling benefits would emerge. It would be worth the effort of writing that extra code and moving some of the heat over to application servers. The thing is, if that’s my problem, I’m in a good spot. It means I’ve got so much demand that I’ve probably got money to pay someone else to do that work. There’s still little reason to start out in the beginning with anything but SQL. You’re likely making more work for yourself in exchange for scaling benefits that you do not need because you are small.

I’m sure there are some applications out there that have few, or even zero, joins. I’m also well aware of many applications where being free of a schema is such a huge boon, it is worth the price of writing more code. These applications exist, but in my experience they are rare. Almost every application I can conceive of is naturally relational in some way.

If anyone out there can explain what I am missing, I will be very thankful. Until then, I’ll be using MySQL for almost everything. It’s easy. It works. It’s fast. I don’t need the super data integrity, or user unfriendliness, of Postgres. And while it may not scale as easily as others, it scales much larger than any application I have ever had to deal with. If I am ever so fortunate to have to scale large, then I’ll make a move. Until then, I’ll save myself the trouble.

Posted in Technology | Tagged | 1 Comment

Not Implementing Features Is Hard

Posted on June 24, 2010 by apreche

Robert O’Callahan, a Mozilla developer, has made a blog post explaining why Firefox omits certain features. Based on the headline, I thought this was going to be discussing the evils of feature creep. That is not the case.

Sometimes we take heat for deliberately not implementing certain Web platform features. Three significant examples are SVG Fonts, WebSQLDatabase and H.264 in <video>.

…

But I honestly think we’re doing the Web a service by resisting these things.I think not implementing certain features is one of the hardest and most altruistic things we do.

Update I should also point out that we’re not just exerting “stop energy” here; for each of these features that we’re not adding, we are promoting superior alternatives — namely WOFF, IndexedDB, and WebM.

– Not Implementing Features Is Hard

I’ve been using Firefox since it was called Phoenix. In those days, your only real browser choices were Internet Explorer or the full Mozilla. Mozilla was a better browser, and even had a crude tabbed browsing implementation. The problem was that it was big and slow because it wasn’t just a browser. It also included email, newsgroups, composer, and more. Imagine if all the different parts of Microsoft Office were a single program. That was Mozilla in those days.

Over time Phoenix, and then Firefox, gained popularity for three reasons. It removed all those features that most people didn’t want or use. This resulted in vastly increased performance and stability that other browsers had a hard time competing with. It removed even more features that some people actually did use, but made them available by extensions. Lastly, they added and polished new features that most users actually were using, such as tabbed browsing.

There was one more thing Firefox did that was very important. Most web sites actually worked in it. This was a time when many sites were built only for Internet Explorer. People would begrudgingly continue to use IE simply because some web site required it. Because most sites worked in Firefox people were able to switch. This is important because it shows that people are more loyal to web sites than web browsers. People will switch to a better browser, but only if all of “their” sites work in it. This is why businesses with Internet Explorer specific intranet apps stick with it, even when the IT staff knows that better browsers exist. The web apps take precedence.

Today we have some browser features like SVG Fonts, WebSQLDatabase, and H.264 video. If Firefox wanted to make an argument that they shouldn’t included these features at all, because of feature creep, that’s actually cool. That’s the philosophy of Firefox. Unless it’s for every user, put it in an extension or a plugin. Keep the browser small.

Too bad that’s not what they’re doing here. They are supporting what they feel are “superior” alternatives. Those alternatives may indeed be superior for legal reasons, or even technological reasons, but that doesn’t mean jack or shit. Web sites are already using these features that Firefox is not including, and the more sites that don’t work in Firefox, the less people will use Firefox.

As a web developer, Firefox is the go to browser. I’m using it right now. Why do I use it? It’s because it has Firebug. I can’t do my job without Firebug. The webkit developer tools are getting a lot better. Most developers I know only use Firefox because there is some extension they need. They really want to stop using Firefox, but can’t. Chrome has extensions now, and most of the important ones are being ported. There’s less and less tying people to Firefox, and if apps work better in webkit browsers, people are going to switch away. It might take some time, but it will happen.

What bothered me most about this article was that he says that they’re doing the web a service. I can safely say, they are doing the web a huge disservice. Let me use something people haven’t paid much attention to as an example. Behold the HTML5 audio tag. It’s just like the video tag, but for audio. Can you guess which browsers support which audio codecs?

Codec support in modern browsers
Browser Ogg Vorbis MP3 WAV
FireFox 3.5 ✓ ✓
Safari 4 ✓ ✓
Chrome 3 (beta) ✓ ✓
Opera 10 (beta) ✓

– Native Audio in the browser

This chart can easily be redone to show support for video codecs, fonts, etc. Just about any new HTML feature follows this same pattern. Yes, all browsers are equally to blame here. If Safari supported Ogg, we would be all set. If Chrome supported wav, that would also solve everything. It takes just one to break the dam.

That being said, I don’t know why none of them do it. The browser that supports the full set will be the browser that works with any web app. Thus, it will be the browser that dominates. Firefox could be that browser, they just have to give up on their philosophical bullshit. Web developers will have peace of mind. The web will become a much better place because there won’t be all sorts of hacks in web applications to switch between different codecs depending on user agent.

There is something to be said for leaving out features completely for the sake of performance and stability. The extension philosophy has got Firefox to the place it is today. Even so, making sure every web site works in your browser is priority number one. And sometimes adding a feature, like tabbed browsing, is beneficial to everybody. These things Firefox is holding out on should be included straight up. I’m starting to think maybe it would be a good idea for someone to create a Firefox fork that has all these features that Mozilla is refusing.

Well, it’s no skin off my back. If they continue to hold out, Chrome and others will just eat their lunch. They say they don’t care about market share, so that’s totally ok by them. It makes me wonder. If they don’t care about market share, why did they always so proudly advertise their download figures?

Posted in Technology | Tagged | Leave a comment

Something Deeply Wrong With Chemistry

Posted on June 23, 2010 by apreche

Over on the Chemistry Blog, Mitch has posted a professor’s letter to a doctoral student. After reading the letter, I can’t even fathom what kind of horrible person the professor is. I imagine that if I spoke to this professor, I would not be able to refrain from punching them in the head.

In addition to the usual work-day schedule, I expect all members of the group to work evenings and weekends.

– Something Deeply Wrong With Chemistry

Sadly, this problem is not unique to chemistry. It’s not even unique to academia. It’s a problem all around the world in a wide variety of employment scenarios. I have encountered many of these scenarios in my lifetime, and they all have upset me a great deal. Allow me to share a few examples that I have seen.

The one that bothers me the most is that restaurant wait staff in the United States are not considered hourly workers. They are paid less than minimum wage, and it’s totally legal. This is why in the US we always tip the wait staff at least some amount, where in other countries you only tip for excellent service. Without tips the wait staff are underpaid. If you don’t tip, you get blamed for being a bad person. Well, the real bad person are the legislators who don’t correct the law. If wait staff were paid minimum wage, we could reserve tipping for excellent service. Prices would go up on menus, but customers will pay the same if they leave no tip.

Another similar scenario is the summer camp counselor. They are also not hourly workers. They get paid a flat fee for the entire summer. A really good camp counselor at a fancy sleep-away camp might make a few thousand for eight weeks of camp and one week of pre-camp. The US minimum wage is presently $7.25. 9 weeks times 7 days times 24 hours times $7.25 is closer to $10,000. Yes, camp counselors do not really work 24/7, and they get a few days off. Even so, most are horrendously underpaid.

One issue that really bugs me is child labor laws. Yes, of course we need these laws to prevent children from being exploited. However, these same laws also sometimes result in minors being unnecessarily restricted. I know many teenagers who would willingly do more work, and make more money, but the labor law prevents it because they are not eighteen. And no, the extra work wouldn’t interfere with homework. These extra hours are spent idling at home playing video games.

Lastly, let us look at countries as a whole. In Europe, many countries have a minimum of thirty days paid vacation for all employees, period. In the US, you are lucky to get ten. The number you get will inch up by one or two every year, assuming you stay at the same job. Switch jobs and you’re back to square one.

This is just criminal. There are people who live to work, but most people work to live. They only work to get money they need to survive in this world, and they would rather not be there. With so few vacation days, you can begin to question the point of living. People need vacation time to actually enjoy the life they have earned. The only reason I haven’t traveled the world already is not due to lack of funds, but due to lack of vacation time. It doesn’t even need to be paid vacation. Just let me leave for a few weeks and promise me that I will still have my job upon return.

Lastly, I must point out the horrible abuses laid upon some salaried employees. If you are paid hourly, or by commission, you might think that a salary is great. You get paid the same no matter what. Yes, that swings both ways. The employer can use threat of termination to get a salaried employee to work extra hours for no additional compensation. Someone with an hourly wage will typically receive time and a half, or some such. Not so for the salaried. They can make you stay late to finish the job for not a single dime extra.

Yes, the labor law is complicated because different jobs are different. Truck drivers, for example, have a completely different structure to their employment than stock brokers. The labor laws for one would be broken in many ways if the same exact rules were applied to both.

Even so, there needs to be some leveling of the board. Some basic principles can be applied equally to all employment. Nobody should face any consequence or demerit for leaving work, if they have put in eight hours that day. Anyone who freely chooses to put in more than eight hours per day should receive some form of extra compensation. Everyone should receive a minimum of thirty days per year vacation time. Even if you are not an hourly worker, your wages, not including tips, shall not be under what the minimum wage would be for the same number of working hours. Rules like these, and many others, should apply to every employment scenario, even the graduate students working in the lab. We already have a few good laws on the books, but we need quite a few more.

There are some people who love their work so much, they would stay there all day and night. Just because they would willingly sacrifice their health and well being, does not mean that everyone else should be forced to do the same. Give them the freedom to work their brains out, if they so desire, but do not force it on the rest of us. Many people look down on companies using off-shore labor, where practices are very bad. Well, there’s little room to talk if similar practices are widespread and legal right here at home.

And you know what, we have an unemployment rate in this country that is rising. If people no longer work extra hours, employers will have to hire more people to get the same work done in the same amount of time. I don’t think decreased profits from having some extra employees is going to hurt the economy all that much. We can just pay the workers with cuts from the CEOs salary.

Posted in Society | Tagged , | 1 Comment