Sad Goodbye

After viewing the The Darjeeling Limited last Friday night, we dined at a favorite Indian restaurant. On the way out, we bumped into a dear friend I hadn't seen since May. Exuding life, an incredibly wonderful human being with a brilliant mind that offered so many contributions to humanity. Recently married...

Forty hours later, I'd incredulously listen to a mutual friend informing me his life had slipped away in a horrible accident.

Goodbye, Steve. You'll be sorely missed.

ETBG

More old Everything But the Girl...

Strangers

Strangers by the Kinks appears in both Rushmore and The Darjeeling Limited.



Artist: The Kinks
Song: Strangers
Disc: Lola versus Powerman and the Money-Go-Round, Part One
Year: 1970

Lone Tree

In spite of efforts and investments in DSLR photography, this shot taken in 2001 with a vintage Kodak point-and-shoot remains my most favorited Flickr photo. Better equipment will give you better results, but, still, being in the right place at the right time is more than half the battle. These days I seem to get most of my good shots from my pocket camera, because I usually have it with me.

Learning from Plants Redux

In June, I offered a few ponderings on the notion of learning from plants--especially in reference to computer science and optimal algorithms (e.g., searching).

Wired has an interesting article today on 'plant intelligence':

"SESTO FIORENTINO, Italy -- Professor Stefano Mancuso knows it isn't easy being green: He runs the world's only laboratory dedicated to plant intelligence.

At the International Laboratory of Plant Neurobiology (LINV), about seven miles outside Florence, Italy, Mancuso and his team of nine work to debunk the myth that plants are low-life. Research at the modern building combines physiology, ecology and molecular biology.

'If you define intelligence as the capacity to solve problems, plants have a lot to teach us," says Mancuso, dressed in harmonizing shades of his favorite color: green. "Not only are they 'smart' in how they grow, adapt and thrive, they do it without neuroses. Intelligence isn't only about having a brain.'"

continued

The Ultimatum Game

Although rational choice theory has found its way into many useful applications, it's too simplistic to explain many of our decisions, especially when issues related to fairness come into play.

The latest issue of Science has a bunch of papers on the psychology and neuroscience of human decision-making:

"One focus of Game Theory is strategic bargaining behavior; the Ultimatum Game (UG) (6) is often used to examine responses to fairness. In the UG, two players must divide a sum of money, with the proposer specifying this division. The responder has the option of accepting or rejecting the offer. If the offer is accepted, the sum is divided as proposed. If it is rejected, neither player receives anything. If people are motivated purely by self-interest, the responder should accept any offer and, knowing this, the proposer will offer the smallest nonzero amount. However, this Nash equilibrium prediction is at odds with observed behavior, and the modal offer is a 50/50 split. Further, low offers of less than 20% of the total amount are rejected about half of the time (6). Thus, people's choices in the UG do not conform to a model in which decisions are driven by financial self-interest, and neuroscience has begun to offer clues as to the mechanisms underlying these decisions."

Alan G. Sanfray. Social Decision-Making: Insights from Game Theory and Neuroscience. (link)

(via Mind Hacks)

Giants

If you're in tech and you've got a penny burning a hole in your pocket (Amazon used), here's a book recommendation:



1998. Interviews with Steve Jobs, T.J. Rogers, Steve Case, Scott Cook, Michael Dell, Andy Grove, Bill Gates, Bill Hewlett and others.

Question: "If finding A players is so important, how can you tell who is an A player and who isn't?"

Steve Jobs: "That's a very hard question. Ultimately there are two paths. If a candidate has been in the workplace for a while, you have to look at the results. There are people who are good on paper and talk such a good story but have no results behind them. They can't point to breakthroughs are successful products they shipped and played an integral part in. Ultimately the results should lead you to the people. As a matter of fact that's how I find great people. I look at great results and I find out who was responsible for them."

(As obvious as such a strategy may seem, it's amazing how few people follow it.)

Facebook = $15B?

I've no expertise whatsoever in valuing companies, but...

I've seen repeated stories on the Net valuing Facebook at 15 billion dollars thanks to a naive running of numbers disclosed with the recent Microsoft deal--and every time I've seen the figure, I've found myself rolling my eyes and whispering "Shyeah..."

Former executive editor of Harvard Business Review Nicholas Carr writes:

The "$15 Billion" Nonsense.

So... all I want to say is...

Yeah, what he said.

On my mind

This may only be of interest if you're a fan of Everything But the Girl or Tracey Thorn. (Although, the reverb of the guitar is pretty cool.) Stumbled onto it on YouTube. Rare, vintage EBTG.




Artist: Everything but the Girl
Song: On my mind
Disc: Pillows & Prayers
Year: 1982

The Darjeeling Limited

Finally made it to The Darjeeling Limited last night, and we both loved it. The Metacritic ratings run from 100 down to 25. Perhaps a good percentage of friends and family would agree with the 25, but I can't imagine any Anderson fan not loving it, and I enjoyed it much more than Life Aquatic.

When it comes to creating films, Anderson's his own man. I don't believe he aspires to make the film some critics damn him for not making. When that's the case, much of the criticism seems undue. One of the things for which I respect Roger Ebert is his insistence on evaluating a film in reference to that which it aspires to be.

On Darjeeling, Ebert writes: "I said the movie meanders. It will therefore inspire reviews complaining that it doesn't fly straight as an arrow at its target. But it doesn't have a target, either. Why do we have to be the cops and enforce a narrow range of movie requirements? Anderson is like Dave Brubeck, who I'm listening to right now. He knows every note of the original song, but the fun and genius come in the way he noodles around. And in his movie's cast, especially with Owen Wilson, Anderson takes advantage of champion noodlers." (3.5/4 stars)

This Note's For You

Last week, I found myself reading an NYT article on MTV executive Judy McGrath. One thing that struck me was PERSONAL HERO: Neil Young.

I love Neil too. Besides his music, it's his uncompromising vision and refusal to sell out. If you've never seen it, what follows is his amusing 1988 music video This Note's For You in which he mocks commercial promotions done by fellow 80s music stars.

The Costs of Mistakes

Often I see systems and individuals go to great expense in order to avoid making mistakes, but I don't think enough thought goes into weighing the cost of avoiding a mistake with the cost of the mistake itself. Sometimes it's much more efficient to simply accept a scenario in which mistakes happen and correct them where and when they occur.

If mistakes are easily detected and corrected, preventive measures may amount to not only an enormous waste of time but an enormous opportunity cost, because all that time spent on preventive measures might be directed toward more productive ends.

Exacerbating the situation is the question of expected costs. When preventive measures are a given, their expected costs and actual costs over a period are one and the same. Often the mistakes targeted by preventive measures are improbable and occur with extremely low frequency.

Even if the cost of the measure makes sense given the cost of the mistake, the overall costs may make no sense at all when the situation is assessed over a significant period of time.

Specialization vs Generalization

From time to time, I find myself arguing both sides of the fence when it comes to specialization vs generalization and software teams; but most of the time, I wind up arguing in favor of specialization.

With his pin factory example, Adam Smith made a classic observation on the power of specialization in Wealth of Nations:

"I have seen a small manufactory of this kind where ten men only were employed, and where some of them consequently performed two or three distinct operations. But though they were very poor, and therefore but indifferently accommodated with the necessary machinery, they could, when they exerted themselves, make among them about twelve pounds of pins in a day. There are in a pound upwards of four thousand pins of a middling size. Those ten persons, therefore, could make among them upwards of forty-eight thousand pins in a day. Each person, therefore, making a tenth part of forty-eight thousand pins, might be considered as making four thousand eight hundred pins in a day. But if they had all wrought separately and independently, and without any of them having been educated to this peculiar business, they certainly could not each of them have made twenty, perhaps not one pin in a day; that is, certainly, not the two hundred and fortieth, perhaps not the four thousand eight hundredth part of what they are at present capable of performing, in consequence of a proper division and combination of their different operations."

Dentists and Highways

A notorious problem in software development is estimating how much time is required to complete a particular software project. I've written an awful lot of software over the past couple decades, complex systems, commercial applications that have been quite successful, etc. In the process, I've received a number of kudos from management on the accuracy of my estimates.

What's my secret? I don't think I have one. If anything, I'd say experience is key--and, perhaps, the ability to relate the level of complexity and uncertainty of something I've never done with something I've done before. I imagine a carpenter who hasn't built a particular cabinet does the same thing--he comes up with an estimate by relating the the cabinet he's asked to build with an essentially similar cabinet he's built during a previous commission.

I don't see how experience can be removed from the equation. I think it's crucial. Asking a carpenter to estimate the costs of fixing a car is as much an exercise in absurdity as asking a dentist to estimate the time needed to pave ten miles of highway.

When you remove the critical component of experience, estimation becomes a meaningless crapshoot, which is why I find this How Good An Estimator are You? quiz silly.

Ship of Fools

From World Party's excellent 1987 release...



Artist: World Party
Song: Ship of Fools
Disc: Private Revolution
Year: 1987

Exploring Space

Abstract: The point of this post is looking at user interfaces as a means of exploring n-dimensional spaces.

One of the most popular means of converting a color photo to a black and white photo is via an adjustment filter known as Channel Mixer. The user interface (Photoshop below) allows one to specify weights for each of the color channels to produce grey values that are linear combinations of red, green and blue.



I've found this approach problematic for a long time for a few reasons.

First, there are four controls but essentially three degrees of freedom; i.e., if grey = weight_constant * (weight_red * red + weight_green * green + weight_blue * blue), it's clear the same effect could be achieved by widening range of the weights associated with each color and dropping the weight_constant term.

Generally, it seems, the weights associated with the color channels are first adjusted and then the overall brightness is corrected once a suitable combination of colors has been selected.

When you present a user with a set of controls such as the sliders in Channel Mixer, in many respects, you're offering an opportunity to explore space. The space can be any number of dimensions depending on how many controls exist. With three sliders for red, green and blue, it's a three-dimensional space.

In the case of Channel Mixer, the addition of a constant percentage doesn't really add another dimension to the space--it simply expands the size of the search space, because it amounts to moving along a vector in 3-space defined by the red, green and blue weights.

When we frame the problem as one involving space exploration in 3-space, interesting questions come to mind. For example, consider the dimensionality of our input devices. For searching, the mouse is a useful input device, but it's 2D at best; and sliders are essentially 1D controls, because they involve movement along a line (hence, you need three sliders to navigate a 3D space).

What is the best possible way to map the 3D search space to inputs from the user?

Hmmm...

One question we can ask is whether or not there's a region in the search space that's of greatest interest to the user. If you'd like to learn more about such questions and how to deal with them, you may want to investigate Principal Component Analysis, which is one approach to reducing the dimensionality of a space while retaining as much salient information as possible. Using such a technique, we might be able to effectively reduce the dimensionality of the search space by throwing away dimensions that aren't interesting to the user.

If we want to give our user the freedom to explore the entire space, we have a couple of options. We can provide three 1-D controls (e.g., Channel Mixer sliders) for space exploration. The problem with this approach is it's really tedious to explore space this way. It's like trying to work on of those skill cranes at an arcade.

Alternatively, we might map our search space into the combination of a 1-D input and a 2-D input--for example, a slider and, say, a rectangle with a little draggable dot. Unfortunately, we can't accept 3D input from our user without providing a 3D device such as a Wiimote or something similar.

Given the prospect of searching a 3D space with a set of controls, another important question ponders the best way to slice it. For example, if we search the Channel Mixer space using a combination of a 2D input and a 1D input, we might use the 2D control to navigate weighting possibilities for red and green and use the 1D control to determine the weighting for blue, but we should ask if this the most useful mapping for the user?

Alternatively, we can slice the 3D search space with a plane from any conceivable direction and then use a 2D control to navigate the plane and a 1D control to move along the surfance normal directions perpendicular to the plane.

In some respects, black and white conversion using Channel Mixer is similar to making lemonade. How can that be? Consider the problem of trying to make the perfect lemonade. It's a liquid created from three ingredients: water, sugar and lemon juice.

Let's say we're going to use from 0 to 2 cups of each ingredient, so we're searching in a 3D space for perfect lemonade, and the space is bounded by 0 to 2 cups in each dimension. If you think about it a bit, you'll realize perfect lemonade won't be a point in this space, it'll be a line.

Huh?

Just for the sake of making thing simple, let's say we find the best lemonade at the point where we have 2 cups of each ingredient. How will this lemonade differ in taste from a lemonade made with one cup of each ingredient?

Answer: It won't.

The 2 cups case is nothing more than a double batch of the 1 cup case. Everything falling along the line going from (0,0,0) to (2,2,2) will be perfect lemonade, and the only thing the distance from the origin determines is how much lemonade you've made.

If you're looking for the perfect lemonade, all of the interesting questions fall in a two dimensional subspace, because what matters are the relative proportions of each component: what % water, what % lemon juice, what % sugar?

On first glance three percentages might not sound like a plane, but there's an important constraint requiring everything to add up to 100%; i.e., if you have 30% water and 30% lemon juice, you have no choice but to have 40% sugar (100% - 30% - 30% = 40%), so even though you have three percentages, you only have two degrees of freedom.

Reconsidering our original lemonade space which merely consisted of three dimensions of varying quantities, we can see that a more useful interpretation of the space is one in which factor the space into a 2D subspace of all possible lemonade recipes and a 1D subspace of lemonade quantity.

The lemonade example demonstrates the sort of considerations one might make in mapping a 3D search space to a set of controls. In the case of Channel Mixer, we can ask ourselves if weights for red, green and blue constitute the best possible factoring of the space defining the color conversion.

What if, like the lemonade recipe, we mapped all possible recipes of red, green and blue to a 2D control and then controlled the overall brightness with a 1D control? This would reduce the problem from four controls down to two*. Such perspectives and questions led to the latest B&W conversion filter in PSP.

Hopefully, some of the thoughts here will inspire new insights, questions and perpectives in others.

fin

*Note: Channel Mixer does have other functions and it allows negative percentages, but these features generally aren't very useful for B&W conversion.

Areté

"The most articulated value in Greek culture is areté. Translated as 'virtue,' the word actually means something closer to 'being the best you can be,' or 'reaching your highest human potential.' The term from Homeric times onwards is not gender specific. Homer applies the term of both the Greek and Trojan heroes as well as major female figures, such as Penelope, the wife of the Greek hero, Odysseus. In the Homeric poems, areté is frequently associated with bravery, but more often, with effectiveness. The man or woman of areté is a person of the highest effectiveness; they use all their faculties: strength, bravery, wit, and deceptiveness, to achieve real results. In the Homeric world, then, areté involves all of the abilities and potentialities available to humans. We can, through the frequent use of this term in Homer's poems, make some tentative conclusions about the early Greek world view. The concept implies a human-centered universe in which human actions are of paramount importance; the world is a place of conflict and difficulty, and human value and meaning is measured against individual effectiveness in the world." ©1996, Richard Hooker

link

Example #23

A new exercise in contrast...



Spent the past week in the enchanting waters of the Caribbean.

My first cruise, beliefs and theories were reinforced and acquired. For example, it's true these excursions turn into horrendous exercises in gluttony. By the end of the week, you and your family have fallen into the epicurean depravity of Henry VIII or an ancient Roman living on the cusp of social collapse. The waiters are brainwashed with the directive "Never say no!" and they obey the order as dutifully as the subjects of the Milgram experiment.

On Day 1, you're still in sync with reality, pragmatically splitting entrees with your spouse. On Day 7, you opt for a full order of prime rib AND a full order of the crab while you scold your children for wastefully taking six pieces of cake simply for the sake of harvesting the frosting. Free insulin should be included.

I've also forged a new hypothesis: the probability of going on a Caribbean cruise without hearing a Bob Marley song is 0% with an infinite number of digits of precision. If you attempt such a thing and don't hear Three Little Birds or Jamming, there's only one possible explanation: hate to break it to you, but you're clinically deaf, dude.

The Sound

My first post-collegiate job: scientific programmer for a Seattle biotech company during the dawn of the 90s. As far as bodies of water go, Puget Sound has its own identity--a distinctive swirl of echoing fog horns, cold grey tranquility and saline mist.

Brain: Left vs. Right

Interesting.

This link contains an animated version of the image above as a brain dominance test. According to the article, right-brained folks see the dancer spinning clockwise while the left-brained perceive counterclockwise rotation.

My wife sees counterclockwise rotation. My natural inclination is clockwise, but I have little trouble refocusing to the counterclockwise perspective.

It's cool, anyhow.

link

The Obviousness Curve

One half-baked idea bobbing in the back of my head is a fuzzily defined notion of an "Obviousness Curve." The basic idea is one of knowledge progressing to a point of peak obviousness; once knowledge increases beyond this point, obviousness declines.

For example, there's a point in every childhood where one is oblivious to the names of colors. With a little education, one learns to recognize red, yellow, blue, etc. At some point, one becomes quite confident in the associations between words and phenomena--so much so that one might ask what fool couldn't recognize the pillow as yellow.

But as one continues to acquire knowledge and learns of spectral power distributions, spectral reflectance, spectral response curves, metamerism, etc., the question of a yellow pillow is much less obvious, and an aswer requires a much more elaborate explanation.

So it is with many things.

( If I've mentioned this before, forgive me--it's been the subject of at least one deleted draft.)

Sometimes dumb is smart...

I've rambled on the subject a bit before, but all too often these days I see people trying to create machines smarter than the people that use them and--all too often--the designers fail miserably.

My wife recently purchased a new microwave and, in spite of its good points, this machine (as did the previous one) tweets periodically and relentlessly until the object of nuke-age is removed. As annoying as this is, it does spare me from a fate from which presumably millions have perished: forgetting to eat.

The underlying algorithm seems to work like this:

State #1: Microwave stops.
State #2: Wait thirty seconds for user to open door.
State #3: Beep and go to State #2.

And this appears to go on ad infinitum. I'm left wondering what sort of person might find this system most useful. My conclusion is the most forgetful individual on the planet. Perhaps there's some cook that will finally realize the memory lapse on the hundredth beep cycle, but the food is going to be cold at that point anyway.

Cast one vote for: If you're smart enough to get something inside a microwave and you're smart enough to program a microwave, you're probably smart enough to determine the optimal time to remove a heated item from a microwave.

Another problem with some smart devices is that they're unpredictable. A while back, I was asked to come up with a response to a "smart" photo retouching tool in a competitor's application. As smart as the competitor was, my biggest beef with the "smart" tool was that it was quite unpredictable. Sometimes it was smart and worked well. Sometimes it was too smart for its own good and the results were random and undesirable. The behavior was even acknowledged in the product Help file with the suggestion that if it doesn't work right the first time, one should just undo the operation and try again--which seems to imply the tool is unpredictable to the point of random.

My answer to the smart but unpredictable tool was a tool that was dumb but predictable. One admitted impetus behind this was the short amount of time I was given to come up with a solution, but another force impelling me to go for the comparatively dumb solution was a strong belief in the importance of creating tools with predictable behaviors.

What seems to be forgotten with many "smart" tools is that humans are smart--much smarter than any so-called "smart" tools devised so far--and because we are so smart, our intelligence can often more than compensate for the dumbness of our tools.

Consider what Michelangelo did with David using tools no smarter than a hammer and chisel. If the sculpture had been attempted with a "smart" hammer and a "smart" chisel that randomly tried to guess the master's intentions, I doubt the sculpture would have turned out as well.

The sculptor's tools are dumb, but they are governed by consistent physical laws, and that makes it possible for the sculptor to learn and compensate for tool deficiencies. When tools exhibit unpredictable random behavior, it is impossible to compensate for deficiencies.

Inspiring me to tap out this post is a run-in with branch prediction on Intel CPUs. Today's CPUs work kind of like little assembly lines. The machine instructions our computers execute go on the equivalent of a little conveyor belt (they call 'em 'pipes').

Usually, this all works fine and well, but problems occur when the CPU comes to a fork in the path of execution. When that happens, the CPU has to guess which path will be taken; once the guess is made, the instructions from the guessed path are transferred to the conveyor belt (this is called "branch prediction").

If the CPU guesses the wrong path, it's bad, because the conveyor belt has to be swept clean and reloaded with the other path. If you want your code to run as quickly as possible, you don't want this to happen--it's an unnecessary waste of time.

One of the rules for guessing the paths least and most traveled used to be fairly simple: assume backward jumps will happen and forward jumps won't. What's nice about this is that it's fairly easy for a developer with a knowledge of the probability distribution of branch cases to write code that optimally leverages this simple and predictable branching rule.

Now consider the behavior for static branches in the Pentium M and Core 2 Duo: "These processors do not use static prediction. The predictor makes a random prediction the first time a branch is seen... There is simply a 50% chance of making the right prediction..." (Agner Fog: The Microarchitecture of Intel and AMD CPUs p. 27)

In fairness, this isn't quite as bad as it sounds because in most cases these CPUs learn from their mistakes pretty quickly when the paths are executed repeatedly inside loops, but I'm still frustrated by the deviation in behavior from predictable to random.

It's hard to write code that compensates for a coin flip. In their repeated attempts to make CPUs run software smarter, CPU manufacturers seem to be making it harder for developers to write smart software. Speaking as a developer, it's much easier to exploit relatively dumb rules consistently applied than it is to try to entertain all sorts of complicated smart variations.

Sometimes dumber is smarter, because dumb is often predictable and, consequently, human intelligence can compensate for it.

This is the Sea

More music... from one performance with a violin to another...

This is from one of my favorite discs, This is the Sea (1985), by the Irish group known as The Waterboys and headed by Mike Scott. They were in town a month ago and, unfortunately, I missed them. In the late 1980s, Karl Wallinger left the group to form World Party.



Artist: The Waterboys
Song: This is the Sea
Disc: This is the Sea
Year: 1985

Back on the Chain Gang

A great live performance by Chrissie Hynde.



Artist: The Pretenders
Song: Back on the Chain Gang
Disc: Learning to Crawl
Year: 1983

Like a Call

Artist: Architecture in Helsinki
Song: Like a Call
Disc: Fingers Crossed
Year: 2004

Hand

Shot taken on a ferry from Seattle to Bremerton this summer.

Faster Cube Root IV

Previously, I created a few posts on the subject of fast cube root calculation (1,2,3); also, n-th root approximation (1). I spent a little time collecting these thoughts into a more cohesive form, which may be found by following this link:

In Search of a Fast Cube Root

Also included is code to approximate cube roots by various means (bit hack approximations, Newton-Raphson, Halley's method) and associated performance results.

Poser

One more shot from the summer. Seattle pier.

Glósóli

Paul Ford at Ftrain:

Scott Rahin told me, “the other day I was watching this video and I just started crying.”
“What was it?” I asked.
“Sigur Rós,” he said. “And before you humiliate me—”
“Was it the one where the kids fly?”
“Yeah.”
“That's the Sophie's Choice of music videos..."

more: $5 Chocolate Bar



Artist: Sigur Rós
Song: Glósóli
Disc: Takk...
Year: 2005

SIMD within a Register

I've been posting a little here and there on bit hacks. A while back I stumbled on a page titled Technical Summary: SWAR Technology by Hank Dietz (1997):

"The basic concept of SWAR, SIMD Within A Register, is that operations on word-length registers can be used to speed-up computations by performing SIMD parallel operations on n k/n-bit field values."

In other words, in the absence of MMX/SSE/etc., which SIMD operations can be done in an ordinary register using bit operations? It's an intriguing puzzle to ponder, especially as 64-bit systems continue to gain ground.

I'm sure it's been done before, but one idea coming to mind is a parallel average. The obvious way to average two unsigned values is (A + B) / 2. Actually, if you want to round up, (A + B + 1) / 2, but what happens if (A + B) overflows? How about adding A/2 and B/2, and if the low bit is set in either A or B, round up by one:

average = (A>>1) + (B>>1) + (A&1 OR B&1);

If we want to average two bytes in parallel using this method, we should be able to do it in a 16-bit word. The only additional thing we need to do is AND with 0xFEFE to mask off least significant bits before the shifft so they aren't shifted into the most significant bit of adjacent bytes:

average_x_2 = ((A&0xFEFE)>>1) + ((B&0xFEFE)>>1) + (A&0x0101 OR B&0x0101);

Of course, this idea readily extends to 32 bits to handle 4 bytes in parallel:

average_x_4 = ((A&0xFEFEFEFE)>>1) + ((B&0xFEFEFEFE)>>1) + (A&0x01010101 OR B&0x01010101);

Likewise, it can be extended to 64-bit for processing 8 bytes in parallel.

Note: I am using OR to indicate a BITWISE-OR because Blogger nukes vertical bars.