My Rambling Thoughts

Quote:

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

Brian W. Kernighan

Quote:

If you ride a motorcycle often, you will be killed riding it. That much is as sure as night follows day. Your responsibility is to be vigilant and careful as to continue to push that eventuality so far forward that you die of old age first.

Unknown source

News:

Date: . Source: .

Top 10 worst technology predictions

"Inventions have long since reached their limit, and I see no hope for further developments,"

Julius Sextus Frontinus,
Roman engineer, 10 AD

"Everything that can be invented has already been invented."

Charles Duell,
commissioner for the U.S. Patent Office, 1899

These two are so funny when you read about them today, but I must admit it is easy to fall into this trap.

Mr Frontinus could be forgiven, because change was slow before 1800. IMO, it was when we learnt how to use electricity in the 1800s that the pace picked up.

(Electricity has always been around. We just learnt how to use it in the last 200 of our 200,000 years of existence.)

Mr Duell had no excuse.

Here's how to phrase it without looking like a dummy a few centuries later: "Current technology — as we know it — is at its pinnacle."

"The Americans have need of the telephone, but we do not. We have plenty of messenger boys,"

Sir William Preece,
chief engineer at the British Post Office, 1878.

"Who the hell wants to hear actors talk?"

H.M. Warner,
Warner Bros., 1927.

"Television won't be able to hold on to any market it captures after the first six months. People will soon get tired of staring at a plywood box every night,"

Darryl Zanuck,
20th Century Fox, 1946.

"The world potential market for copying machines is 5,000 at most,"

IBM executives to the eventual founders of Xerox, 1959.

It sure is hard to predict across a paradigm shift.

"I think there is a world market for maybe five computers,"

Thomas Watson,
chairman of IBM, 1943.

"There is no reason anyone would want a computer in their home,"

Ken Olsen,
founder of mainframe-producer Digital Equipment Corp., 1977.

"No one will need more than 637 kb of memory for a personal computer — 640K ought to be enough for anybody,"

Bill Gates,
Microsoft, 1981.

These quotes are understandable, but many people still like to quote them — out of context — today.

In 1943, computers were special purpose computing devices. There was no commerical market. The first general-purpose computer, ENIAC, was completed in 1946.

(For a long time, I knew what ENIAC stood for — Electronic Numerical Integrator And Computer. It was an irrelevant acronym I had to memorize for inter-school quizzes. I never liked the acronyms, but I did like the history.)

In 1977, home computers were toys — not much more than game consoles. The computers were minis and mainframes. You definitely don't want them in your home. Here is how it sounds like in context: "There is no reason anyone would want a shopping mall in their home." No reason at all. Oops, it already is.

The Bill Gates quote is the most famous one — people are still quoting it to make fun of Microsoft. He said it because he just moved from a 64 kB architecture. 640 kB was 10x as much! It was thought the PC would be obsoleted well before the limit was hit.

Well, Bill Gates was indeed a bit shortsighted. Just 1-2 years later, PCs were shipped with 256 kB and soon the full 640 kB RAM. The ceiling was hit.

What would you have said back then?

Remember, RAM was very expensive in those days. (That's why we have so many fancy data structures. Today, just use array and hashtable.)

Well, things change. To be safe, predict 100x as much. 6,400 kB was not common until the early-90s.

(The 640 MB mark — 100x of 6,400 kB — was reached around 2005, so never say never.)

"Next Christmas the iPod will be dead, finished, gone, kaput,"

Sir Alan Sugar,
British entrepreneur, 2005.

No excuse for this one. It was not just iPod that did the trick, but also iTunes with its $0.99 songs.

(These predictions are quoted from a columnist.)

No oil for you

transport

News: Tanker Glut Signals 25% Drop on 26-Mile Line of Ships

Date: 28 December 2009. Source: Bloomberg.

A 26-mile-long line of idled oil tankers, enough to blockade the English Channel, may signal a 25 percent slump in freight rates next year.

The ships will unload 26 percent of the crude and oil products they are storing in six months, adding to vessel supply and pushing rates for supertankers down to an average of $30,000 a day next year, compared with $40,212 now, according to the median estimate in a Bloomberg News survey of 15 analysts, traders and shipbrokers. That's below what Frontline Ltd., the biggest operator of the ships, says it needs to break even.

This article is about the oil tankers rates, but the real story is about the use of oil tankers to store oil instead of transporting them. (This has been happening since a year ago, but no one paid any attention to it.)

Oil at US$76 is an illusion. It's because the holders of these oil stocks don't want to sell it to you — at this price. They'll rather store it. (Not that there's anything wrong with it in a capitalist society.)

This indicates to me there is lot of excess oil around. Just how long are these people able to hold on to the oil? $40k/day = $14.6mil/year. Google says a supertanker can hold 2 million barrels. That means $200mil if oil is at $100. $14.6mil is 7.4%.

Personally, I feel if these people are able to make oil go beyond, say, US$120 and make a profit, then the consumers are ripped off — again. It's like heads-I-win, tails-I-also-win.

(Now, I like expensive oil — cheap oil encourage wastage. As far as our current understanding goes, oil is a non-renewable resource. Once it's gone, it's gone. This is a problem because our current way of life is powered by oil! On the other hand, I don't like artificially manipulated high prices.)

Car market remains flat

transport

News: No rebound in premium car market despite better economy

Date: 26 December 2009. Source: BT.

AN EXPECTED rebound in demand for premium cars as the economy recovers has not materialised. In fact, some distributors of high-end makes say that the market has softened instead.

'When the good news about the global economy began trickling in from July and August, we expected demand to strengthen and the momentum to build up,' said the boss of a small luxury dealership. 'But that didn't carry through to the end of the year.'

Is this representative of the real economy?

I don't know what to think of the global economy anymore. Governments and Central Banks all over the world have an incentive to paint a bright picture, but it is consumer spending that we must look out for. Are they spending beyond the necessities?

A look at Darker than Black 2 (Gemini of the Meteor)

I find Darker than Black 2 a much weaker show than the first season.

The first season of DtB uses a 2-episode per story format with a 3-episode final story for a total of 25 episodes. The stories are mostly standalone that reveal the world and build up the characters. From the middle part onwards, they also start to build towards the final story.

There are 11 stories and they are all very good. If you like the characters and the world, you'll wish the stories go on forever. But some people don't like this format because they advance the main story very slowly.

The second season has only one main story over 12 episodes. There are several side-stories that are resolved, but you never really feel for the minor characters because they are not built up sufficiently. The side stories feel like time-wasters.

1 – 3 In Vladivostok (Largest port city in Eastern Russia).

Sudden change in life for Suou; tried to escape from everyone. Got caught by Hei. Suou's father and twin brother, Shion, had already made their escape.

Hei lost his powers and Suou gained hers. Mao and July joined them.

4 – 6 In Sapporo (Japan). Training and finally assassination.

Suou learnt martial arts from Hei and practised her marksmanship.

A local boy, Norio, got a crush on Suou and she was affected by it — he reminded her of a boy she had a crush on in her hometown.

7 – 8 Getting out of Hokkaido to Tokyo by car. Got betrayed, got to rescue July.

Ep 8: took a train from Sendai onwards. Got to face Russia pursuers, including her ex-best friend.

9 – 10 Suou searched for her mother — one of the two reasons why she came to Japan.

Ep 10: Suou found her mother, but her mother rejected her. Misaki, the ex-police officer, tracked her down to find out why. It was then that Suou learnt her back-story from her mother's POV.

11 The whole truth.

Suou went to the aquarium planted in her consciousness and met her father there. She was told her back-story from her father's POV.

12 Inside the Gate (a not-of-this-world place).

Suou went inside the Gate and met Shion. Her story was then resolved.

The main story is okay, but it is too long. 12 episodes clocking a total of 4+ hours is just too long. It can be trimmed to 2 to 2.5 hours excluding the side-stories.

Note that the story is about Suou and not Hei, who was our protagonist in season 1.

Why does everyone want Suou?

Everyone meaning CIA, FSB, MI-6, The Syndicate, Section 3 and who-knows-what-else.

Why did Suou's life change so suddenly? She was living in peace for 12 years and all of sudden everyone wanted her?

In the story, Izanami and Izanagi are prophesized to bring an end to the world if they meet, so some parties are trying to prevent that. Others, however, want them for their own purposes — even without the prophecy, Izanami and Izanagi have super special powers.

(You can google about Izanami and Izanagi; they are creation deities in the Japanese mythology.)

Section 3 has Izanami, whom we also know as Yin. This is a minor spoiler, but don't worry, she doesn't do much in the show. Izanagi is Shion.

The prophecy was leaked and hence the whole world became interested in Shion.

Yes, everyone wanted Shion, not Suou. Shion made her his decoy — by lying to her and asking her disguise as him. Still, Shion wanted Suou to reach the Gate, because her destiny could only be fulfilled there.

It was implied that Suou's father (or an ally) tipped everyone off. It doesn't make sense. Suou, Shion and their father could have gone to the Gate quietly before everything started. (As I said, Shion really wanted Suou to go to the Gate.)

The strange thing is, after everyone found that they were pursuing Suou and not Shion, they did not let up. Maybe it is because Shion and his father hid themselves very well — they were spotted, but were never in any danger of being caught.

What does Madam Orellie want?

At the start of the show, Hei was working for Madam Orellie, who asked him to retrieve the meteor core from Dr Pavlichenko (Suou's father). After Hei caught Suou (after mistaking her for Shion), it was then never mentioned again. Note that only Hei was after the meteor core, eveyone else was after Shion.

After finding out Madam Orellie was in fact working together with Dr Pavlichenko, this seemed like a roundabout way to get Hei to rescue Suou.

Also, Madam Orellie later ordered Hei to train Suou so that she could assassinate Izanami. Hei stopped Suou in the nick of time after finding out Izanami was Yin. (Hei, Yin and Mao were partners in season 1.)

But Madam Orellie didn't really want to kill Izanami. In fact, she wanted Izanami and Izanagi to meet up. Madam Orellie really wanted to free Izanami, so again her actions were opposite from what she wanted. Did she count on Hei to find out Izanami's identity and not kill her? That seemed risky.

Pay as you drive

transport

News: Holland's Plan to Tax Every Kilometer Driven

Date: 23 December 2009. Source: Times.

Traffic jams are infuriating wherever you are, but in the Netherlands, they are the source of particular angst. Not only is the densely populated country home to Europe's most congested metropolitan region — the area called the Randstad that incorporates Amsterdam, Rotterdam, Utrecht and The Hague — but many Dutch people live below sea level, making them more than a little nervous about carbon emissions, global warming and the possibility that their country could soon be underwater.

No wonder then that the Dutch government is trying something revolutionary to reduce the number of cars on the road — taxing every motorist who gets behind the wheel based on the distance of their trip and the kind of car they drive. The plan, which was approved by the Dutch cabinet in November and is expected to be implemented in 2012, aims to eventually cut the number of traffic jams in the country in half.

I have a simpler idea: tax the gasoline! Other than that, I like the idea of pay-as-you-drive, as it is pretty fair — for a light/moderate user. (If you drive often, you'll hate it.)

This will work really well to reduce traffic in Singapore. The average yearly mileage is 20,000 km. This translates to 64 km/day (for a 6-day week). How do people manage to drive so much on a 42 x 22.5 km island? It's like everyone is staying halfway across the island from where they work/shop.

I use IE 7 because IE 8 sucks

computer

Someone online remarked that he is still using IE 7 because IE 8 sucked. He also mentioned that he suffered regularly from some sort of iframe attack. This was my response to him:

No it doesn't.

IE 8 finally does CSS 2.1 right. It is able to use the same stylesheet as the other browsers, such as FireFox 3. (No CSS hacks needed, unless you want rounded corners and translucent backgrounds.) And if you need IE 7, it is built into IE 8! Just toggle it. I have no need for a separate IE 7 anymore.

And what's this iframe attack? Does it exploit some buffer overflow to escape from the browser sandbox and execute some native code? Otherwise I don't see how it can compromise your machine. Even ActiveX isn't run automatically anymore. Also, is your IE patched?

Be sure to enable IE's protected mode if you are running on Vista/Win7. It'll limit any damage if compromised — the browser is running in a limited account and has limited access to the registry/file system. To break it, the attacker needs to gain Administrator access — yet another hurdle.

I use Opera for my daily surfing and I turn off Flash/Java. This works well enough for most sites. There is no need to block cookies or disable JavaScript.

The old-is-better-than-new thought is applicable to most mature applications, but browsers are not there yet.

Do I need a 64-bit OS to access >4GB RAM?

computer

Not necessarily. The server versions of Windows since Windows 2000 are able to use all the installed memory. However, due to the Windows memory architecture, a single process can have only up to 2 GB of memory, and all the processes share the same 2 GB kernel memory.

Windows also has a /3GB flag: it gives a single process 3 GB address space and the kernel 1 GB. This is a useful for a large single-process server, such as a mail server, but is a bad idea for a multi-user system.

I don't know how the memory is laid out for Linux, but I had the impression there is no 2-2 split. Processes can use (almost) the full 4 GB memory.

Note that these are "virtual" (or what Intel called linear) memory. The processes/kernel map to the same 4 GB physical memory, because the underlying processor uses 32-bit physical addressing. 2^32 = 4 GB. How can a 32-bit OS access more than 4 GB physical memory then?

Well, Intel introduced PAE (Physical Address Extension) in the Pentium Pro in 1995: a 32-bit linear address is mapped to a page table (or two) that maps to a 36-bit physical address. 2^36 = 64 GB.

In other words, a 32-bit OS can use >4 GB memory if it supports PAE. It's just that the 32-bit processes running on it are not able to — at least not in one unified address space.

Browser shakeout end 2009

Browser market share 2009 week 51

Latest statistics:

  • IE 7/8 has 40% market share. IE 7 is downtrending as more people upgrade from it to IE 8.
  • FF 3.x has 30% share. People are in the midst of upgrading from 3.0 to 3.5.
  • IE 6 is downtrending faster than I expected. It looks like most people who are stuck with it for one reason or another are now using another (non-IE) browser for general surfing, since IE 6 cannot co-exist with another version of IE — the irony.
  • Chrome is rising rapidly at 5.5%.
  • Safari, even as the primary browser on OS-X, only has 3.5% share.
  • Other than Opera, we can ignore all other browsers/versions.

The FireFox graphs show very clearly that users do upgrade to the latest version. FireFox 2.x, for example, has just 1% share now.

But what really stand out is that IE 7 users also do so. Only IE 6 users do not upgrade.

A compulsary review for all Star Wars fans

First, I'll like to state that there are only three Star Wars movies (and the third one isn't very good either). Anyone who tells you otherwise is a liar. But if by some unfortunate circumstances that you have watched some crappy prequels that claim to be Star Wars, you'll love this review of The Phantom Menace.

Go to YouTube, search for The Phantom Menace Review RedLetterMedia and watch the 7-part video review. Be sure to have 70 minutes on hand, cos you won't be able to stop!

Note that the review is not 100% SFW (Safe For Work). The reviewer got issues: he speaks with a speech impediment, he doesn't know his history, he killed his wife in a car accident and he kidnaps hookers in his basement! At the end, he even has a showdown with... well, you got to watch it all.

Despite the reviewer's mental condition, he did a very good review of TPM. :lol: He never once touched on Jar Jar Binks nor Anakin's mother's virgin birth. They are problems, but TPM has got much bigger issues. Instead, he talked about characters and plots and compared them to the real Star Wars movies.

Final Fantasy XIII is too linear

Final Fantasy XIII is out, and early gamers are said to be surprised by how linear it is. I wonder how can it be — Final Fantasy games have always been very linear.

(Note that despite the Final Fantasy moniker, the games don't have anything in common.)

Here is a sample criticism:

The whole game is completely linear until halfway through [approximately 20 hours].

There are hardly any towns.

The party is always fixed. The whole game just repeats the movie-battle sequence over and over [In fact the progression is closer to movie-run-battle-run-movie-boss-movie-run-battle...].

I have no idea if the criticism is valid or not. I'm most concerned about the story. I'll be disappointed if FF 13 has a poor story. I'm no longer interested in Final Fantasy as games, but I'm still interested in the game story. I've watched FF 7, 8, 10 and 12. Yes, watch — people upload their playthroughs to YouTube.

(I didn't watch FF 9 because I've never been interested in it. FF 11 is a MMORPG.)

Watching instead of playing

Games are meant to be played. But games that have a story can be watched. Whether it can be done or not depends how good the story is. Some plots are so paper-thin that they can be explained in a minute. FF stories are pretty decent; they always have plot twists and revelations along the way. (Think "Luke, I'm your father." Oops, I hope I didn't spoil it for you! :-D)

Just to give an idea of a story's complexity and depth: Final Fantasy X has over 8 hours of dialogue and cut scenes — excluding the walking and battles.

The problem with game stories is that there is usually only one main story and a small number of pre-determined endings. This translates to a very linear game. FF 7 uses character backstories and optional side quests, so you get to know more about the characters and the world.

To have a true choose-your-own-adventure game, I feel there has to be two breakthroughs:

  • ability to combine independent movements on-screen, instead of needing to script the entire scene beforehand
  • voice synthesizer to allow generic dialogue

A feather in my cap: preparing my own meals

Self cooked meal

One of my first self-cooked meals. Very simple dishes; almost nothing can go wrong. :-D

I know how to cook a few other dishes, all involving vegetables. That's all I need — I'm a semi-vegetarian!

Eventually, I hope to learn a few more complicated dishes: fish-n-chips and dumplings. Not that I particularly like dumplings, but I can only eat specially prepared ones by my mother (due to my dietary requirements), so it is good to learn to DIY.

Greek letters

We use them all the time, but how well do you know the full set:

Alpha, Beta, Gamma, Delta, Epsilon, Zeta, Eta, Theta, Iota, Kappa, Lambda, Mu, Nu, Xi, Omicron, Pi, Rho, Sigma, Tau, Upsilon, Phi, Chi, Psi, Omega

I have never heard of Kappa, Nu, Xi, Omicron and Upsilon before.

How much to own a car in Singapore?

News: Minimum $7650 family income to own a car comfortably?

Date: 15 December 2009. Source: AsiaOne.

If you're thinking of buying a car but your combined family income is below $7,650, think again.

According to a local website salary.sg run by a finance manager, $7,650 is the minimum income needed to afford a car.

Most families are not able to afford cars because the cost of car ownership does not only include the price but also the miscellaneous fees such as road taxes, car insurance and servicing, said the website.

$7,650 seems rather high, but it is plausible because it is lower than the 38th percentile and 38% of the households own a car.

The PI director is right. You can own a car with $2,000 income — even with $0 downpayment. You just have very little to spend elsewhere.

(A 10-year $50,000 loan at 2.5% is just $521/month. It is affordable at first glance. But don't forget you need to pay for 10 years.)

It is usual to spend $1,200 to $1,600, including the installment, for a typical 1.6L car. As long as you are willing to pay this amount every month, you can technically afford a car.

The installment is usually $600 to $900, so the cost of car ownership can be less than $1,000 for a paid-up car. Unfortunately, most Singaporeans don't get to enjoy this because they always change their cars — 80% of the cars are less than 4 years old.

Fuel cap, at last

transport

News: Malaysia enforces fuel cap on foreign vehicles

Date: 16 December 2009. Source: BT.

The 20-litre fuel cap on foreign vehicles travelling within a 50-kilometre radius from the border will be implemented today, Domestic Trade, Cooperatives and Consumerism Minister Ismail Sabri Yaakob said.

He said the ruling was aimed at checking smuggling of petrol and diesel at border areas until a new mechanism on fuel subsidy was implemented in May next year.

Note that the cap is 20 litres, including the fuel in the tank.

PRs who live in JB won't be able to top-up there anymore. Singaporeans also can't go in to pump cheap fuel. (Until they tamper their meter.)

But will it work? Officially, yes. But this being Malaysia, a black market will spring up in no time. The simplest: just transfer the fuel from a Malaysian car.

Lost in Boon Lay — if such a thing is possible

transport

I wasn't lost, technically; I just couldn't find the place I was looking for.

Singapore is a very small place, so it is just not possible to be lost — you'll find a familiar landmark sooner or later. However, it has many small roads, so it is very easy to miss a place.

Jurong West

A colleague SMSed me to see if I wanted the famous Boon Lay Lei Cha. He was on the way and offered to buy for us.

I wanted to have it, but I was on leave. No problem, I'll just go down to buy it myself (it's 20 km from Toa Payoh to Jurong West). Consulting the street directory, I found that I just need to make a couple of turns (the blue lines) after exiting from PIE. Nothing could be easier.

Well, I spent 20 km rounding the place (the red lines) and two calls to a colleague for directions before I found it.

Where did I go wrong?

First, I did not bring my street directory with me.

Second, I did not prepare a secondary landmark from which I could re-orient myself. In Singapore, it is usual to pick MRT stations, as they are very convenient landmarks. This is usually a waste of time, but is invaluable when it is needed — when you got lost or when the primary road is jammed, for example.

Minimum JavaScript competency

programming

The days of simple JavaScript are long gone. In those days, you could claim to know JavaScript if you could write this:

if(document.field1.value == "") {
  alert("Field is empty");
  return;
  }

Very C/Java like. So many server-side programmers claim they know JavaScript. (Just like they claim to know HTML and CSS. No, they don't.)

Today, you need to know these:

  • A JavaScript framework, say, jQuery
  • Asynchronous code flow
  • Objects (including this)
  • Callbacks
  • Closure

Don't let these scare you off; they are easier than they seem. For example, if you understand this code — all of it, including the parameters — you are more than halfway there:

$.ajax({
  type: "get",
  url: "/script",
  dataType: "json",
  async: true,
  success: function(data, textStatus) { ... }
  });

In just a single function call, no less. After that, try this:

(function fn() {
  $(".cls").toggleClass("cls2");
  setTimeout(fn, 500);
  }) ();

(Note that toggleClass is a jQuery 1.3 function — always know your API.)

The code won't work right over time; its purpose is to illustrate some JavaScript concepts. After you figure out what it does (not difficult), the next step is to figure out why. :-D

Anything else?

You need to know about null, undefined and what is loose-typing.

You also need to know the API for Object, Array, String and Date as they are used quite often. (Some familiarity is good enough. I often google to get the parameter ordering right.)

Lastly, you also need to know regular expressions. Simple RE will do. Perl-like RE is not welcome, actually — they are hard to read, hard to get right and hard to maintain.

There, it's not that difficult, right?

Well, these are the minimum required to do a Web 2.0 app. There is a bit more to learn, but if you can handle these, you will have no problem with the rest.

Rich Dad, poor dad

finance

A few years ago, Rich Dad, Poor Dad was all the vogue. My sister bought it and recommended it highly, so I started reading it... and never got past chapter one.

You see, the author talks about his two Dads, a rich neighbour and his "poor" real Dad, and he prefers the rich neighbour. Now, I'm not filial or anything (you can ask my father :lol:), but this just struck me as wrong.

Anyway, I searched for reviews and I found:

  • His "real-life" story is fake.
  • His advice is incorrect or (partially) illegal.
  • His book became famous only after the MLM crowd hyped it up.

Extracted from a review:

Rich Dad, Poor Dad contains much wrong advice, much bad advice, some dangerous advice, and virtually no good advice.

I only have one thing to say: the book should be placed under fiction, not personal finance.

Deadlock: waiting for myself

I pulled the latest code to test, but the build was broken. I checked every now and then but it was still broken after two hours. How to work like that?

Then someone emailed me and asked if I were fixing the code break?

Huh? :???:

I then looked at the error more carefully and found that I was the one who broke the code!

A lesson in search and meta-search

I wanted to download a bunch of videos from YouTube. The most primitive method I used is copying the video file from the IE cache, but it only works reliably for one video at a time. (And don't ever exceed the cache size!)

But there is no need to use this method any more. There are websites that can give the download link. I tried one of them. It works most of the time, but it sometimes require several refreshes before the link shows up. (I wonder why.)

Finally, I got fed up and looked for another tool. I found a bookmarklet that exposed the very-slightly higher quality MP4 file instead. It works in Opera. Life is good.

(A bookmarklet is a bookmark that is actually JavaScript. It runs in the context of the page where you click the bookmark.)

I used it for a while, then I realized it only recognized a very specific video format. In fact, it ignored a higher quality format if it is available. That's not what I want!

YouTube video formats

A detour. Commonly found YouTube video formats:

  • SD (fmt 5): 320x240, FLV, H.263, MP3
  • MP4 (fmt 18): 480x360, MP4, H.264, AAC
  • LQ (fmt 34): 320x240 to 640x360, FLV, H.264, AAC
  • HQ (fmt 35): 480p, FLV, H.264, AAC
  • HD (fmt 22): 720p, MP4, H.264, AAC

(It is said format 18 is specially created for iPhone, which can only play MP4.)

LQ is just below the minimum watchable quality. SD is unwatchable. To illustrate the difference: a 10-minute video clip can be 15MB for SD, 25MB for MP4, 70MB for HQ and 160MB for HD!

A well-known "trick" to get higher quality video is to append &fmt=22 to the end of the video URL. As you can see, this selects the HD version if it is available. Unfortunately, YouTube does not support graceful degradation, so it does not select the HQ version if that is the best quality available.

User JavaScript

I could modify the bookmarklet, but I googled some more and found someone has already written a pretty comprehensive download tool! It is not a bookmarklet anymore — it is User JavaScript.

(User JavaScript is like bookmarklets, except it runs automatically when you visit specific websites. Like bookmarklets, it runs in the context of the current page — this is a powerful concept.)

This particular script does two things:

  • It shows all the supported video formats and allows you to click to save them.
  • It goes through all the video links on the page and shows their quality!

Although it is written for FireFox, it runs perfectly in Opera. (And the script is well-written too.)

Life is pretty good.

Not so good, maybe

By this time, I had already downloaded quite a number of video files. I was curious if I had chosen the videos correctly.

It was tedious to open each link one by one to judge the video quality, so I only did a limited search based on two simple criteria: (i) the files must be complete (80 to 110+ per set), (ii) compare 3 to 5 uploaders.

Now with this tool, I could quickly find which links have the best video quality by simply going through the search results.

And I found that I did not download the videos with the best quality. So I had to download all over again. Oh well.

A lesson in search

Why did I miss the videos on my first search?

Well, the videos with the best quality were buried deeply because they were relatively new uploads, whereas the older ones had been viewed "a million times".

This is definitely a "popular-get-more-popular" syndrome. Take note of it during searching, because there may be buried gems a few pages deep.

Extensible tools

I'm able to use Opera to download the videos concurrently. It works, but it also saturates my bandwidth. Unfortunately, it does not have a bandwidth throttle feature.

It also does not notify me when the current batch of download is done. (I usually download 4 to 8 files at a time.)

FireFox comes to the rescue!

Although FireFox also cannot do these out-of-box, it has extensions that can do so:

  • GreaseMonkey to run User JavaScript
  • Bandwidth throttler
  • Notify on download complete

Life is good.

The takeaway

User JavaScript is very cool. Users can extend the webpage's functionality themselves. This should be very useful for stores, auction and search sites. It sounds like Web 3.0 to me.

Extensions allow the user to customize/extend the browser in all sorts of manner. Nothing comes close to FireFox for this. And its extensions are usually a package of JavaScript and XML! (Although there are binary extensions.)

This is the age of JavaScript, no doubt about it.

Amazon 1, Kinokuniya 0

I ordered three books from Amazon, including Joel on Software and The Old New Thing. I ordered them on the same day a short novel I wanted came out. I didn't order it because I thought Kinokuniya may have it (I thought it would be a same-day worldwide release). I was wrong. Kinokuniya could only get the novel after it was released in the US.

Well, I received the books from Amazon after three weeks, yet the novel is still not available from Kinokuniya.

I wonder about the future of B&M bookstores.

A few times I wanted to look for books armed with only very vague descriptions, like a partial title, or an English title when looking for the Chinese version. Guess how the Customer Service searched? Through Amazon. (And sometimes Google.)

North Korea devalues, hmm

finance

Four interesting news for the past two weeks:

  1. Dubai World asks to delay debt repayment for 6 months. This could trigger CDS and also cause the US carry trade to unwind. While this could be a non-event, it could also be a trigger, if you think that today's market is a farce.
  2. North Korea devalues at the tune of 100:1 and a cap of 150,000 won plus 300,000 won in the bank. (150,000 won is just US$60 at black market rates.) In North Korea, you better not be a saver.
  3. Japan starts to sell its treasury bonds. It increases demand for the US$ and makes it more expensive. This could cause the US carry trade to unwind — anything big-scale to make the US$ more expensive can do so; just waiting for a trigger. Is this one of them?
  4. AmTrust, a US regional bank, filed for bankcruptcy protection to avoid the fate of earlier banks (like Washington Mutual last year) after being seized by FDIC. WaMu shareholders were wiped out when FDIC took over the bank and sold it to JPMorgan Chase. (The shareholders have much more at stake than the US$250k insured limit.)

All pretty interesting in their own ways. I'm not surprised by (1), am amused by (2) [until another country tries it], wondered just how much more bonds the Treasury has to soak up for (3) [China has stopped buying], and laughed at (4). It is a fine example of "unintended consequences".

What good is a camera that doesn't zoom?

photography

Leica recently released the X1 camera: 12Mp APS-C CMOS sensor, 24mm f/2.8 lens (equivalent to 36mm).

A friend asked me this question: "What good is a camera that doesn't zoom?"

My friend was puzzled because a fixed lens sounded so restrictive. He gave an example of shooting tigers in the zoo.

Well, I feel this is the kind of camera Leica would release: a small and light camera with an excellent fast lens for street/available light photography — and a price to match.

And that is its purpose: street photography. This camera (or rather, the 35mm focal length) is for people who like to get close to their subjects. It certainly cannot take sports nor wildlife, but that is not what the camera is meant for.

Let's compare the X1 to an SLR with 28-70/2.8 lens. At first glance, you may think that the SLR is superior. After all, you can zoom from 28mm (wide angle) to 70mm (mild telephoto) and you have f/2.8 throughout.

That is, until you try it.

First, the SLR is big and bulky. You'll get tired in no time. Worse, you are very conspicuous. Street photography means photographing people in their natural states, meaning you have to be discreet.

And the X1 fulfills this role much better than the SLR.

Zooming with your feet

But no zoom? Well, there is a much better alternative to zoom: walk around and take the pictures from different spots. Zoom makes you lazy. You stay in one place and the pictures all look the same.

It is a common advice to new photographers to use the 50mm lens to improve their eye for photography. It helps that the 50mm lens gives very good quality pictures — and is almost always the cheapest lens in the lineup.

This is good advice, but not many people stick with it, because a single focal length is so restrictive that people just give up, claiming "there are so many pictures I can't take with this lens!" It is when you try to force it through that you experiment with new ideas and learn to use the environment to your advantage.

So, I told my friend, "the day that you understand, is the day you become a photographer." :lol:

(And also, I suggest using the 35mm lens. It is easier for beginners because it is slightly wider. 50mm is neither here nor there, so it is quite difficult to use well. And once you are past the learning stage, you'll find that the 35mm lens is pretty useful as a street photography lens — just about as discreet as you can be with an SLR. The 50mm lens, you will need to pair with another wide-angle lens, either the 24mm or 28mm lens.)

Note that on SLRs with 1.5x crop factor, the lens you want is the 35mm lens, not the 50mm lens. The fact that people still recommend the 50mm lens blindly shows that they don't really understand the principle. (And if you take my suggestion, it's the 24mm lens that you want.)

Expenses for November 2009

finance

TBD.

How Acquire should be played

Acquire '93

My copy of Acquire has finally arrived after a month-long wait!

The box is much bigger than I expected! (It was just under the limit for normal mail and the seller could only wrap a thin piece of wrapper around the box. The box took a little damage as a result.)

Yes, 3D plastic pieces... this is how boardgames should be played!

Acquire has a long history, going back to 1962. It has at least 15 different releases over the years. The latest one (2008) is a budget edition aimed at new gamers. It works, but it feels a bit abstract placing cardboard chits instead of blocks/buildings.

Most serious gamers would seek out the previous 1999 Avalon Hill large box version. It had high production values, but it was expensive and hence sales were poor. When it went OOP, the price went up because it sort of became a "collector's item"!

Personally, I think the '93 release by Schimdt Spiele is nicer, that's why I bought it instead.

(Sidenote: the seller is a stamp collector and asked me to send the stamps back — he has even prepared an envelope for it! I'll see if there are any interesting local stamps to send back to him. :lol:)

Code reuse vs do-it-yourself

Code reuse is good. I like it. However, more often than not, I prefer to implement key functionality myself. Third-party code is seldom an exact-fit and takes time to learn and modify. I might as well DIY.

Then, I read this quote from Joel Spolsky and I finally understood:

If it's a core business function — do it yourself, no matter what.

The core functionality is what differentiates your software. If you use third-party code, it is more generic and easier to duplicate.

(Of course, you can argue what is core or non-core. Also, third-party code refers to code you use as a black box. If you understand the code thoroughly and have made extensive customizations to it, it is no longer third-party code.)

Joel also went on to note something very important. He basically says you should do it only if you can do it better!

This is very important. I always do some research before I DIY; I will use third-party code if I cannot do it better.

Things mobile phones will make obsolete

Things someone on the net claimed the mobile phone will make obsolete:

  • Phone boxes
  • Wristwatches
  • Bedside alarm clocks
  • MP3 players
  • Landline home phones
  • Compact digital cameras
  • Netbooks
  • Handheld games consoles
  • Paper

I doubt they will disappear entirely, but most people will be happy with what the mobile phone provides and only the "geeks" will seek out the specialized versions. A niche market, in other words.

It is just like IE and the other browsers — most people use IE because it comes with Windows and don't bother with other browsers. (This reminded me how Sun forced Microsoft to remove MS-JVM from Windows XP. It certainly didn't help Java gain market share.)

I have some comments on two of the points: netbooks and paper.

I doubt mobile phones will replace netbooks. Rather, netbooks will replace notebooks. In other words, notebooks are in the danger of being obsoleted, not netbooks.

(As an example, rather a one-size-fits-all notebook, I am considering buying a notebook not exceeding 1.3 kg and a "server" desktop PC. The PC will serve mainly as a repository.)

Paper is getting obsoleted with consumers, because they will rather view stuff from their mobile phone. The only reason they print is because they have to — many B&M businesses still require B&W copies. We have to make it easier for them to print such stuff.

Google Chrome Frame vulnerability

computer

Microsoft found a XSS vulnerability in Google Chrome Frame (the Chrome browser plugin for IE) and reported it "responsibly" — in Google's own words — to Google so that they could fix it.

Many people think it is ironic, because of this:

Internet Explorer 8's famous XSS filter can be exploited to perform successful XSS attacks against web sites which would be otherwise safe. In other words, XSS "protection" is helping XSS attackers, oh the irony.

Microsoft has not fixed it after months. Google knows about it and put X-XSS-Protection: 0 in the HTTP header to turn the filter off. No wonder some people asked Microsoft to put their own house in order first.

Beyond irony, this just goes to show that no software is bug free. The more things you add, the bigger the attack surface.

Google uses an auto-push update policy: it updates the browser/plug-in without informing you. It seems intrusive, but people seem to accept it so far. (This policy should simplify life for developers — just test the latest version.)

A page that cannot be printed

computer

I tried to print two pages by specifying the absolute page numbers. Nope, the wrong pages came out. No problem, I went to the correct section and specify the logical page numbers. Still the wrong pages came out. I tried a few more combinations and still failed.

I threw the printer away in my rage.

(I was pretty irritated for the day, and it was the straw that broke the camel's back.)

Even TLS can be exploited

TLS (the successor to SSL) is the cornerstone of Internet security. Now someone has found a new exploit — not in the encryption, but in the protocol. This exploit is known as the TLS Renegotiation Attack and lies somewhere between a MITM (Man-In-The-Middle) and CSRF (Cross-Site Request Forgery).

(This is the third known exploit in the past two years. To be fair, all are easily patched.)

Well, as least the encryption is still sound. Some encryption that has been proven weak over the years: CSS (used in DVD), WEP, WPA-TKIP (demonstrated just a few weeks ago).

(Weak in the sense that the data can be decrypted trivially — within a minute or so — without the key.)

Microsoft violates GPL, sort of

Microsoft pulled one of its tools offline after someone detected parts of it was copied from GPL code. Apparently someone disassembled the .NET code to trace a bug and found that it looked familiar.

Microsoft said the code was delivered by a third-party, but they still accepted responsibility for lapse in review.

Microsoft also decided to release the source to the tool — since this is what GPL requires. Cut-and-paste code from the Internet is serious business.

Ask the right questions

programming

The Guerrilla Guide to Interviewing (version 3.0)

(October 2006)

A motley gang of anarchists, free-love advocates, and banana-rights agitators have hijacked The Love Boat out of Puerto Vallarta and are threatening to sink it in 7 days with all 616 passengers and 327 crew members unless their demands are met. The demand? A million dollars in small unmarked bills, and a GPL implementation of WATFIV, that is, the esteemed Waterloo Fortran IV compiler. ...

As chief programmer of the Festival Cruise programming staff, you've got to decide if you can deliver a Fortran compiler from scratch in seven days. You've got a staff of two programmers to help you.

Can you do it?

"Well, I suppose, it depends," you say. ...

On what?

"Um, will my team be able to use UML-generating tools?"

Does that really matter? Three programmers, seven days, Waterloo Fortran IV. Are UML tools going to make or break it?

"I guess not."

OK, so, what does it depend on?

"Will we have 19 inch monitors? And will we have access to all the Jolt we can drink?"

Again, does this matter? Is caffeine going to determine whether you can do it?

"I guess not. Oh, wait. You said I have a staff of two programmers?"

Right.

"Who are they?"

Does that matter?

"Sure! If the team doesn't get along, we'll never be able to work together. And I know a few superstar programmers who could crank out a Fortran compiler by themselves in one week, and lots of programmers who couldn't write the code to print the startup banner if they had six months."

Now we're on to something!

I like the way Joel tells it because sometimes you have to think a bit to see what is important and what is not.

Interesting Flash security vulnerability

computer

Now, a vulnerability in Flash. Basic premise:

  1. Find a server that doesn't check uploads properly
  2. Upload a SWF disguised as JPG
  3. When the victim visits the page, the SWF is loaded and has access to all his site data (same-origin policy)
  4. The SWF also has access to the attacker's server if he has configured his crossdomain.xml policy (which he would have done, obviously)

(If it is a social site or forum, you don't even need steps 1 and 2.)

Never underestimate just how vulnerable our computers are.

I routinely turn off Java and Flash for normal web surfing — not for security, but speed. It looks like it is the right thing to do! :lol:

I don't really like web-based Flash contents because they integrate poorly with HTML. The only reason I keep Flash around is to watch YouTube videos. (I only install Flash for IE, and although IE is much more secured than it used to be, its reputation is in ruins and I only use it for testing purposes.)

And just in case you are wondering, Adobe has indicated they would not fix this vulnerability.

The scale is not linear

programming

Hitting the High Notes

(July 2005)

The real trouble with using a lot of mediocre programmers instead of a couple of good ones is that no matter how long they work, they never produce something as good as what the great programmers can produce.

...

The Creative Zen team could spend years refining their ugly iPod knockoffs and never produce as beautiful, satisfying, and elegant a player as the Apple iPod. And they're not going to make a dent in Apple's market share because the magical design talent is just not there. They don't have it.

The mediocre talent just never hits the high notes that the top talent hits all the time. The number of divas who can hit the f6 in Mozart's Queen of the Night is vanishingly small, and you just can't perform The Queen of the Night without that famous f6.

...

It's not just a matter of "10 times more productive." It's that the "average productive" developer never hits the high notes that make great software.

The big team vs small team approach. Unfortunately, I must disagree with Joel. It is very rare to meet a really good programmer, much less two of them on the same project.

Rewriting from scratch? Think again

programming

Things You Should Never Do, Part I

(April 2000)

Netscape 6.0 is finally going into its first public beta. There never was a version 5.0. The last major release, version 4.0, was released almost three years ago. Three years is an awfully long time in the Internet world. During this time, Netscape sat by, helplessly, as their market share plummeted.

It's a bit smarmy of me to criticize them for waiting so long between releases. They didn't do it on purpose, now, did they?

Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

...

We're programmers. Programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. We're not excited by incremental renovation: tinkering, improving, planting flower beds.

There's a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:

It's harder to read code than to write it.

...

The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they've been fixed. There's nothing wrong with it. It doesn't acquire bugs just by sitting around on your hard drive.

...

When you throw away code and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work.

You are throwing away your market leadership. You are giving a gift of two or three years to your competitors, and believe me, that is a long time in software years.

...

It's important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time. First of all, you probably don't even have the same programming team that worked on version one, so you don't actually have "more experience". You're just going to make most of the old mistakes again, and introduce some new problems that weren't in the original version.

I have rewritten code enough times to know what Joel is talking about. It may sound strange, but the older code is often more robust. It is ugly due to all the special handling put in over the years. But after you replicate them in your shining new code (after overlooking them in your design — of course), I bet your new code doesn't look that nice either.

Now, this doesn't mean no-rewrite-at-all-cost. The key is: leverage, change incrementally and maintain backward compatibility.

An inch at a time

programming

A game of inches

(June 2007)

Commercial software — the kind you sell to other people — is a game of inches.

Every day you make a tiny bit of progress. You make one thing just a smidgen better. You make the alarm clock default to 7:00am instead of 12:00 midnight. A tiny improvement that will barely benefit anyone. One inch.

There are thousands and tens of thousands of these tiny things.

...

And as you fix more and more of these little details, as you polish and shape and shine and craft the little corners of your product, something magical happens. The inches add up to feet, the feet add up to yards, and the yards add up to miles. And you ship a truly great product. The kind of product that feels great, that works intuitively, that blows people away.

This article from Joel Spolsky struck a chord for me.

Writing software is a bit like sculpting. You create the general form first and then you carve out the details — which is the time-consuming part.

Font size at 10.5pt now

It took me over an hour to track down the long-standing off-by-one pixel bug in my homepage's tabbed interface. This bug exists only in Chrome and Safari (basically WebKit browsers); it looks fine on IE, FireFox and Opera. However, I could not find any documented off-by-one pixel bug for Chrome.

After many attempts to determine the cause, I was ready to give up and looked for a CSS hack to target only WebKit browsers — as a short-term solution — but I couldn't find one! (Actually, I found a few, but they are too fragile to use.)

Then, I thought this was such a common technique, so I searched for tutorial webpages. They looked fine. I took a look at one webpage's CSS and found that we have the same CSS! What gave? I could not tell, so I copied the HTML/CSS to my website, which promptly failed to work!

It took me another minute to unravel the mystery: my font size was too big and caused its container to expand by one pixel — it was not a true off-by-one pixel bug. (I'm not surprised this happens only on WebKit browsers. They render fonts slightly differently.)

This is a good enough reason for me to set my default font size to 10.5pt (14px). Previously I didn't set it, and most browsers default to 12pt (16px), which is slightly too big.

Despite using relative font sizes everywhere else, I had to update them. 80% at 16px is acceptable, but is too small at 14px. I changed it to 85%. Relative font sizes only work to a certain extent.

So you can fail inspection...

transport

It is time for the annual ritual to get my papers in order again — inspection, insurance and road tax.

I went to my usual inspection centre, which was supposed to be super-lenient. Well, not so.

The inspector took one look and said, "failed!" — due to my bald rear tyre. My heart sank; I was thinking of having to pay the inspection fees again. Luckily, he said it's free! That's nice.

So you can fail inspection. :-D

I went to my usual workshop to change the tyres. I feel their business has changed in the past two years: they have a huge number of quite-new bikes with for-sale signs.

Anyway, the mechanic asked me if I wanted to change my rear brakepad. I said it should still be fine. It looks like now even mechanics have to push-sell.

My thoughts were confirmed when I came back to collect my bike — they changed the air filter as well! I had totally forgotten that my bike has one, so I thought he was talking about the oil filter. But he repeated "air filter" slowly a few times so that I could understand.

I was thinking how come they went ahead to change without informing me? Perhaps they couldn't reach me — I did have one missed call from them.

If I remembered my bike has an air filter, I would have changed the part myself. It would only be around $10 instead of the $20 they charged me. But nevermind, I overlooked it and the air filter was somewhat dirty and clogged. (Maybe that's why my FC has been going up.)

So later I went back for re-inspection. The inspector (a different one) only inspected the rear tyre — he just took one look at it. My original front tyre wasn't bald, so I could have just swapped the front and rear tyres and pass the re-inspection. :lol:

Adding/updating a SQL record in one step

programming

I always found it strange why SQL does not have an integrated INSERT/UPDATE. INSERT fails if the record already exists. UPDATE fails if the record does not exist.

(On the same note, I'm also puzzled by the huge difference in syntax for INSERT and UPDATE. SQL was obviously designed by an ex-COBOLer.)

Luckily, MySQL allows an integrated INSERT/UPDATE:

INSERT INTO tbl (key, field1, field2)
  VALUES (keyv, value1, value2)
  ON DUPLICATE KEY UPDATE
    field1 = VALUES(field1),
    field2 = VALUES(field2);

Or this:

REPLACE INTO tbl (key, field1, field2)
  VALUES (keyv, value1, value2);

This seems to be a closer fit, but take note that this is a DELETE-ADD operation.

And just for completeness sake, you can also do this:

INSERT IGNORE INTO tbl (key, field1, field2)
  VALUES (keyv, value1, value2);

This will insert the record just once. Subsequent ones are ignored and the operations are considered successful.

Don't send me a PDF file!

computer

A colleague said JavaScript was a rubbish language due to its lax syntax and weird parsing. He posed this question to me:

var x = 1;

function fn() {
  alert(x);
  var x = 2;
  }

fn();

What would be displayed? "Undefined", I told him. Wrong, he said it is 2. Now, I'm very sure it is undefinedbecause I have tested it before. Well, I tested it again and it is still undefined.

(If you understand why the answer is undefined, you would have understood how JavaScript handles var.)

In any case, my colleague has missed the woods for the trees, so I asked him to watch Douglas Crockford's presentation JavaScript: The Good Parts.

He then told me he was unable to play it, so I gave my standard advice: use Media Player Classic. He refused to install it because it was an unsigned untrusted third-party application of unknown origin.

I don't want to make too much fuss over it, because everyone has a list of software they will install on their PC. For example, I don't have Acrobat Reader on my PCs — all except one. So, don't send me PDF files; I'm not able to open them unless I'm at my one PC that can do so.

Anyway, the video is on YouTube, so I'll just ask him to watch it from there. I hope he has the Flash Player on his PC.

Still buying books?

programming

I bought two computer books today:

  • Masterminds of Programming
  • More Joel on Software

Masterminds of Programming is a collection of interviews with 17 computer language designers. I'm interested in computer languages, so this is right up my alley. Unfortunately, JavaScript is not included.

More Joel on Software is from Joel's blog. I like what he writes. It is always very insightful.

I wanted to buy Joel on Software and The Old New Thing, but they weren't in stock. (Amazon comes to the rescue!)

I haven't bought any technical computer books in a long long time. I saw three books that piped my interest:

  • Windows Internals
  • Advanced Windows Debugging
  • C++ Coding Standards: 101 Rules, Guidelines, and Best Practices

I like to read how a real OS is designed and implemented. Windows Internals shows you the plumbing for Windows XP/Vista. I like this stuff, but it is often so dry that I can't finish the book! (Not in one go, anyway.) I may borrow this one from the library.

Advanced Windows Debugging is recommended as a well-written book, but it is so technical and different from my area of work that it is useless to me.

My first thought on C++ Coding Standards: do I need another C++ book? I have way too many already! Well, C++ is a language full of pitfalls, no wonder there are so many intermediate-level books to show you how to avoid them. It remains to be seen if I'll buy this book.

A comparison of my computer language books:

  • JavaScript: 1
  • C: 4
  • x86 assembly: ~6
  • C++: ~12

These are mostly intermediate-level books. Beginner-level books should be borrowed from the library because you outgrow them very quickly. As for reference books, I prefer to use online API reference.

C++ is not difficult to pick up, but it is difficult to use effectively. (The scary part is that you don't realize you are using it inefficiently.)

Update: I found that I already have C++ Coding Standards! :duh: It looks like I never found the time to read it.

(It'll be a while before I get back to C++. These days, I do the bulk of my programming in JavaScript and PHP.)

The desktop is the new root directory

computer

The desktop is the new root directory (from the user's point of view). It took a long time for me to accept it, but I finally did.

(If you look at the Explorer on XP, the Desktop is the root element, not My Computer.)

Storing files on the desktop is much more obvious to a new user — they can see their files directly. They don't need to learn about directories and how to use Explorer.

Expenses for October 2009

finance

TBD.

Merging identical SQL tables

programming

Suppose you have a general-purpose log table. It is tempting to put all logs (transactional logs, events and debugging messages) inside. However, it is better to define a separate tables for them. In particular, this makes it easy to clear out the debugging messages without any side-effects.

What if you need to combine them to get a clear picture of the events?

SELECT * FROM tbl1
UNION ALL
SELECT * FROM tbl2
ORDER BY time_ms;

We cannot order by id, but the tables probably have a timestamp field that we can use to order the records.

(Note that we cannot distinguish the order for inter-table sub-ms operations, but this is usually good enough.)

Updating multiple rows in SQL

programming

It is easy to update multiple rows with the same value:

UPDATE tbl SET field = value WHERE cond = 'something';

It is possible to update multiple rows with different values, but the syntax is a bit convoluted:

UPDATE tbl SET
  field = CASE cond
    WHEN 1 THEN 'x'
    WHEN 2 THEN 'y'
    WHEN 3 THEN 'z' END
WHERE cond in (1, 2, 3);

The WHERE clause is to narrow down the rows that you are interested in. Without it, SQL may need to go through the entire table!

You can update multiple columns simultaneously too:

UPDATE tbl SET
  field = CASE cond
    WHEN 1 THEN 'x'
    WHEN 2 THEN 'y'
    WHEN 3 THEN 'z' END,
  field2 = CASE cond
    WHEN 1 THEN 'x2'
    WHEN 2 THEN 'y2'
    WHEN 3 THEN 'z2' END
WHERE cond in (1, 2, 3);

Multiple inserts in one query

A related task is multiple inserts in one SQL query. This is inefficient:

INSERT INTO tbl (field1, field2) VALUES('x', 'y');
INSERT INTO tbl (field1, field2) VALUES('x2', 'y2');
INSERT INTO tbl (field1, field2) VALUES('x3', 'y3');

MySQL allows this:

INSERT INTO tbl (field1, field2)
  VALUES('x', 'y'),
  VALUES('x2', 'y2'),
  VALUES('x3', 'y3');

Very straightforward, but apparently this is non-standard, so it is best to isolate this query so that it can be modified for different databases.

Programming effort

It is only slightly harder to generate the more efficient query. We typically issue the single queries in a loop. Now, we just use the loop to construct the combined query and issue it after the loop.

The takeaway

If I'm not wrong, both techniques are only useful for small number of records, because there is a limit to SQL's input buffer and processing space.

It is still possible to make it work: instead of 100 single queries (which is slow) or 1 combined query (which may not work), you break it into 10 combined queries.

The difference between specs and code

programming

Taken from an online blog:

"Writing specifications is like writing a novel. Writing code is like writing poetry."

No wonder it is hard to do both at the same time. I found that I do both better if I concentrate on one or the other. (It takes some time to be "in the zone".)

One thing I found about writing specs is that it can reveal corner cases or subtle interaction bugs, which you then need to go and fix.

The third crash

computer

I have a bad run of HDs this year: my notebook's HD gave up after giving about two weeks advance warning. This is my third HD crash this year. (Two HDs dead and one HD corrupted.)

The HD was already not working properly from day one. It made clicking sounds periodically (every 15s or so). Based on past experience, such sounds mean the HD is on its last leg.

Well, the HD worked fine for almost six months. Then one day, it suddenly stopped momentarily and emitted a beep. I thought it died, but it spinned up immediately. I instantly stopped whatever I was doing and backed up all my data.

The HD continued to work normally. I heard the third beep just a few days ago and went into daily backup mode — I was prepared for an imminent HD failure.

Despite the backup, I still lost some data as I did not back up my entire user account. (I don't know why I didn't do that.) I lost my bookmarks (again!) for sure. I think I lost one to two documents and some recent saved games.

Now I'm absolutely convinced of the need to replicate data — never rely on just one HD.

64 is a lot of bits

programming

Now that we are on the verge of moving to 64-bit platforms, it is time to revisit computer language support for it.

Computers excel at fixed-size storage. The computer languages are designed with it: 32-bit integers and 64-bit floating point numbers.

But with 64-bit processors, it is time to have better variable-size types for one simple reason: 64 is a lot of bits and we don't usually need all of it.

A 64-bit floating-point number can store a 53-bit integer. If you need more than that, you probably want a "BIGINT" integer to avoid overflowing anyway.

Addresses

Addresses are constrained to 48-bits (on desktops anyway) — 256 Terabytes. We can use the unused 16-bits to store something else. If we limit ourselves to 40-bit addresses (1 Terabyte), we have 24-bits for other purposes. 24-bit is sufficient to store 16 million values. For 32-bit pointers, we typically have just 1-2 bits to play around.

Tagged pointers and pointer-with-embedded-data may make a comeback.

int, long and void *

Because 32-bit platforms have been around for so long, everyone takes it for granted that sizeof(int) == sizeof(long) == sizeof(void *).

int is supposed to be the "natural machine word", but it has to be 32-bit in the real world. When it was 16-bit in DOS days, programs were very fragile because they overflowed the 16-bit range easily, especially for multiplication. So people tend to use long a lot in those days.

And now, moving from 32 to 64 bits, if int is changed, it would break a lot of binary interfaces and file structures. We should have concrete types for such purposes.

Because long is often treated interchangeably with int, it is also defined as 32-bit on a 64-bit platform. I feel it is wrong, but we already have long long that means (at least) 64-bit. Mark my words, it will be kept at 64-bit.

void *, being a pointer, has to be 64-bit. The biggest impact will be to callbacks that take an integer argument. We can no longer fit a pointer into it. This is why callbacks should take in a pointer argument rather than an integer.

(Note that it has to be a void pointer — which is guaranteed to be able to contain all other kinds of pointers.)

Minimizing HTTP traffic

programming

When you have a bandwidth quota, you'll learn to watch your usage:

  1. Use Cache-Control so that the browser will not even make the request.
  2. Use Last-Modified and/or ETag to avoid resending still-current contents.
  3. Use gzip for text transfers.

This is easy for static pages — Apache does it for us (once properly configured). What about dynamic pages?

There are three kinds of dynamic pages:

  1. Infrequent updates (wrt views), so output can be cached
  2. Frequent updates and changes can be easily detected
  3. Frequent updates and changes cannot be easily detected

Infrequent updates

This is almost like a static file, except that if the cache file is out-of-date, we need to regenerate it.

Easy to detect changes

This is for cases where the SQL query is something like:

SELECT * FROM tbl WHERE id > $last_id;

If there is no new data, then the result is empty.

Not easy to detect changes

This is for cases where the SQL query is something like:

SELECT * FROM tbl WHERE field = "something";

We cannot easily tell if the result is different from our last query.

I googled and the simplest solution seems to be: generate the output anyway, then take the MD5 and compare it with the ETag. If they match, then just return 304.

The beauty of using the MD5 is that you need it for the ETag anyway, so it is not wasted effort. Note that this method is not 100% foolproof, but it sure is much easier than if you try to do it the "Computer Science" way (comparing the previous and current logical records).

In this case, you still spend the processing power, but at least you save some bandwidth.

Long polling

Another technique to reduce network traffic is the so-called long polling technique. Basically, the server holds the connection until there is new data or timeout.

The disadvantage is that a worker thread is tied up on the server-side, and there is only a finite number of them.

Simple ETag for jQuery Ajax

programming

I found to my surprise that jQuery 1.3 does not support ETag. It does support Last-Modified. Looking through the source, it seems easy to add the support to it, but I decided to write a wrapper instead:

$.ajax({
  url: ...
  ifModified: true,

  beforeSend: function(xhr) {
    if(etag_cache[this.url])
      xhr.setRequestHeader("If-None-Match", etag_cache[this.url]);
    },

  complete: function(xhr, textStatus) {
    var new_etag = xhr.getResponseHeader("ETag");
    if(new_etag)
      etag_cache[this.url] = new_etag;
    }
  });

A few notes:

  • ifModified must be set. If not, jQuery will not check for response 304 (not modified).
  • The server must send back Last-Modified for response 200 (ok) or jQuery will treat it as 304!
  • error, not success, is called with textStatus = "notmodified". If we define this callback, we need to check for this case to avoid a false-negative.
  • There may be XML or Javascript warnings/errors.

Searching for a match

programming

This code baffled me:

function hasCode(code) {
  var list = "105 111 146 161 174";
  return list.indexOf(code) >= 0;
  }

Obviously the programmer came from a very string-based programming background.

In this particular case, it won't have false-positive because code is always 3-digits. But it is very easy to miss this prerequisite.

There are two ways to do it in Javascript. Using a hashtable:

function hasCode(code) {
  var list = { 105: true, 111: true, 146: true,
   161: true, 174: true };
  return list[code];
  }

(This does not return a true boolean, but it works if we test for truthfulness.)

Or searching an array:

function hasCode(code) {
  var list = [ 105, 111, 146, 161, 174 ];
  return $.inArray(+code, list) >= 0;
  }

Note that we use jQuery because IE's Javascript does not support indexOf() for arrays.

In both cases, there are subtle number-string type conversions happening.

The take

People carry over their past programming practices into new languages, where they are no longer applicable. It takes a while to "think native".

Boolean math 101

programming

Many programmers are not familiar with boolean. How hard can a type with just two values be? However, I frequently see this type of code:

if(v != 0)
  return true;
else
  return false;

A person who understands boolean will do this:

return v != 0;

Too trivial? Another example:

switch(v) {
case "A":
  widgetX.show(true);
  widgetY.show(true);
  break;

case "B":
  widgetX.show(true);
  widgetY.show(false);
  break;

case "C":
  widgetX.show(false);
  widgetY.show(false);
  break;

default:
  break;
}

Why not this:

var state = {
  "A": [ true, true ],
  "B": [ true, false ],
  "C": [ false, false ]
  };

if(state[v]) {
  widgetX.show(state[v][0]);
  widgetY.show(state[v][1]);
  }

If you understand this, you have not only understood boolean, but hashtables as well.

Windows and UTF-8

programming

Windows NT went with UCS-2 (Universal Character Set) in 1992 because it was assumed a 2-byte character set could contain all possible characters. The entire Win32 API was revamped with 2 calls: one taking ASCII string and the other Unicode string.

It was a huge exercise. Even worse, there are now 3 sets of string functions in the C library: ASCII, Unicode, and "auto-select". Also, the C compiler still defaults to ASCII strings, so all literal strings must have the L prefix: L"This is a string". It was a mess.

Just one year later, the UTF-8 standard was released. The good thing about it is its awesome backward compatibility: it works with (almost) all existing string functions. Needless to say, today almost everyone uses UTF-8 instead of UTF-16 (the variable-length successor to UCS-2 after they found they could not fit all characters in 64k space after all).

This is water under the bridge, of course.

(Note that people still prefer to use UCS-2 instead of UTF-16 when they can get away with it. Fixed-length characters are much easier to work with.)

On strings

Due to variable-length encoding, I don't think it is a good idea to treat strings as dumb character arrays. Strings should be first-class objects — which they are, except for C.

The memory representation is then immaterial. It is only when you pass the value to an external API that you need to specify the encoding.

Another tough P6 math question

News: Parents up in arms again over PSLE Mathematics paper

Date: 10 October 2009. Source: CNA.

The first thing her son did when he came out from the Primary School Leaving Examination (PSLE) maths paper on Thursday this week was to gesture as if he was "slitting his throat".

"One look at his face and I thought 'oh no'. I could see that he felt he was condemned," said Mrs Karen Sng. "When he was telling me about how he couldn't answer some of the questions, he got very emotional and started crying. He said his hopes of getting (an) A* are dashed."

Notice the word again? It's the same every year. There is always at least one super tough question.

And the way to solve it is always the same: either use algebra, or be really good at the box model.

Arc 2 of Spice and Wolf 2 is disappointing

The second arc of Spice and Wolf 2 has very little economics. After a very long setup, we finally see Lawrence struck a too-good-to-be-true deal with Abe, a female trader in disguise.

The town Lawrence was in, Renos, halted the sales of furs. After a long meeting, they finally decided to allow it to be sold — but only in cash.

Abe's plan was to raise a large sum of capital quickly by using Horo as a collateral (whom she would pass off as a noble), buying up the furs and reselling them further south for a 2x profit.

Also, the owner of inn that Lawrence and Abe stayed in, Howard, offered to sell Lawrence his shop because he wanted to go on a pilgrimage.

Although Lawrence found Abe suspicious after doing some investigation, he still went ahead with the deal — the profit was too enticing. Horo was worth 2,000 Trenni (60 Lumione).

The deal did not go smoothly, of course. When Lawrence went to Abe with the money, Abe tried to kill Lawrence — twice. Lawrence also figured out Abe's game when he saw that she had an equivalent sized money bag.

Despite Abe's claim to be a statue figure trader, she was in fact smuggling salt for the local Church. Abe was the one who hatched the idea to allow cash-only fur trades to the bishop. Unfortunately, the bishop wanted a much bigger business partner, and not a small-time trader like her. Abe was angry and still wanted to proceed with her plans on her own — in secret, of course. (No one can afford to offend the Church in the middle ages.)

She clearly did not lack cash, so I don't know why she involved Lawrence.

In the end, Abe beat Lawrence unconscious and took his money, but she also left him the deed to the inn. Lawrence eventually used the deed to redeem Horo.

I have many questions, but no answers.

If the inn is worth 2,000 Trenni, why not just sell it?! Why go through such a convoluted scheme?

And what about Howard? Did he get Lawrence's share instead? Abe and Howard seemed to be partners.

The season ended with Lawrence and Horo leaving the town — to look for Abe? The story apparently carries over to the next novel.

Some thoughts on boardgame ranking

The current top 10 boardgame ranking, from BoardGameGeek:

1 8.25 Agricola
2 8.24 Puerto Rico
3 8.14 Power Grid
4 8.12 Twilight Struggle
5 8.10 Through the Ages: A Story of Civilization
6 7.98 Space Hulk (3rd Edition)
7 7.97 Dominion
8 7.96 Le Havre
9 7.93 Tigris & Euphrates
10 7.92 El Grande

Some people make a big deal of the top 10 rankings, but I feel they are all very close (statistically speaking).

Ticket to Ride is ranked 52 with 7.45. Note that it is only 0.5 delta away from the #8 game. Robot Rally is ranked 104 with 7.21. Less than 0.3 delta from TTR. The 500th game has a score of 6.52. That is just 0.9 delta away from TTR.

If you ask me, I'll say the ranking is meaningless. Better read the reviews to see if you like the game.

Joined the slashed top group!

transport

I went to my car this morning and found it was slashed. :cry:

MX5 slashed top

I wonder if it's mendable?

Judging by the position, the vandal seemed to be after the cashcard, but it should be plain that there was no cashcard.

The door was not opened and nothing was taken (there is nothing of value to take anyway).

Or, maybe someone was upset I took his regular no. 66 lot for 2 days without shifting the car. (You'll never know — given how mentally unbalanced some drivers are.)

I put this notice up:

MX5 offer reward

I don't think it'll help, but I wish there's more we can do. Vandalism in the MSCP is like hitting below the belt — our cars are basically sitting ducks there.

I also made a routine police report. This is just to add to the regular statistics. I didn't call the police right away because there is no way they are going to catch the vandal. Then, I realized that if everyone think like that, then everyone would think there is no crime!

Looking for an OOP German game

I'm looking for an old out-of-print German boardgame. The logical place to look for it is in Germany, of course! Germany is too far away, but not amazon.de.

I usually try eBay or Amazon, but they usually don't have non-English titles.

I found a few sellers right away. The price is surprisingly low for Gebraucht - Sehr gut copies, which translates directly to Used - Very good.

Let's email the sellers: :-)

Hi, I am interested in this game. However, I have a few questions.

1. I would like to know if this is the 1993 version of Acquire by Schmidt Spiele (just to be sure).

2. I would also like to know if you ship to Singapore.

I am sorry I only understand English. :-)

Thanks to BabelFish, which is all Greek, er, German, to me:

Hallo, bin ich an diesem Spiel interessiert. Jedoch habe ich einige Fragen.

1. Ich möchte wissen, wenn dieses die Version 1993 von erwerben durch Schmidt Spiele ist (gerade sicher sein).

2. Ich möchte auch wissen, wenn Sie nach Singapur versenden.

Ich bin traurig, dass ich nur Englisch verstehe. :-)

I sent both in the same mail.

I bought one OOP boardgame from eBay Germany two years back. At that time, I sent only in English, and got back broken English. But the transaction was eventually successful.

Let's see if I manage to get this one.

Expenses for September 2009

finance

TBD.