A simple SVD calculator

The latest experiment, a Simple SVD calculator, is the result of an attempt to port code from the NIST Template Numerical Toolkit (TNT) to JavaScript (specifically, the Singular Value Decomposition calculation code). Seems to be working. Standard disclaimers apply.

After writing a few functions to support vectors and matrices in a manner as syntactically close to the NIST source as possible, things went across pretty cleanly with minor modifications. (Basic rule of porting: if it's complicated, make as few changes to the source as possible.)

link

All Sparks

I stumbled on a short article on The Editors, but I think the band comparisons missed the mark. If these guys remind me of anyone, it's Echo & the Bunnymen.



Artist: The Editors (MySpace)
Song: All Sparks
Disc: The Back Room

Simple programming lab

When it comes to leisure, I tend to get too many irons in the fire. (So much to learn, so little time.) I'm trying to devote more time to learning more web technologies and languages I'd like to know better (lately, JavaScript).

I'm also trying to come up with a decent means of introducing the kids to programming. Added to that, I'm still working on refreshing and improving my skills in linear algebra. (And there are other projects as well.)

Lacking the time to do it all, I try to come up with projects that fulfill multiple objectives simultaneously. I'll admit doing linear algebra in JavaScript is a little sadistic--but... whateveh... as long as it works.

When I was introduced to Lisp, one of the things that appealed to me was the ease with which one could go between data and code. It's a snap to write code that generates code, self-modifying code, code that converts input to code, etc. because code and data amount to the same thing.

Similar powers exist in JavaScript thanks to the eval( ) function, which lets you evaluate a string and execute it as code.

Along the lines of what I did with the Excanvas example, I've thrown together what I'm calling "a simple programming lab." The system includes an instance of a simple class for output:

out.clear( ) clears the contents of the output window.
out.write( ) writes data without an advancing to the next line.
out.writeln( ) writes data and throws in new line (like Pascal).


Very simple stuff, really. It's one of the simplest programming environments I can imagine.

link

Totoro

Create a stage name by translating "Quincy Jones" to Japanese and you wind up with "Hisaishi Joe".

Born Mamoru Fujisawa, Joe Hisaishi is responsible for the beautiful music found in Hayao Miyazaki's films.

A Million Random Digits

Lately, I've spent some time investigating and reviewing various means of generating random numbers, which have all sorts of applications in science and computing.

My Internet searches left me once again stumbling upon "A Million Random Digits with 100,000 Normal Deviates," a longtime publication of the RAND Corporation that amounts to a long list of random numbers for use in research.



The thought crossed my mind to add an Amazon review for this book, something along the lines of "A real page turner! Unpredictable!"

Clearly, I wasn't the first person with this idea. Here are snippets from customer reviews:

"A truly amazing genre-breaking work of art unlike any that has ever been or ever will. I was captivated from the moment I opened the cover until the extremely suspenseful moment I turned the last page. With that said, I was a little disappointed that 71602 was knocked off by 92937 just as the plot was unfolding, but the arrival of 96240 really got my blood pumping and I just couldn't put the book down from that moment on. "

"The book is a promising reference concept, but the execution is somewhat sloppy. Whatever algorithm they used was not fully tested. The bulk of each page seems random enough. However at the lower left and lower right of alternate pages, the number is found to increment directly."

"This one has a very unpredictable plot, sublime character development in a style that stubbornly defies any sort of development in its rare and iconoclastic brilliance, and is told remarkably with numbers instead of letters. Take, for example, this passage on page 202, "98783 24838 39793 80954". I'm speechless."

"I was moved to tears when I saw little 47774. So pure. So innocent. So palindrome. There's no way you can't fall in love for little 47774."

(In all, there are 80 customer reviews...)

Life on Mars

Too much has been said about the Pink Zune. On to more music...

My first exposure to Brazilian singer Seu Jorge singing Portugese renditions of Bowie came with Wes Anderson's The Life Aquatic with Steve Zissou.

If you can catch him in concert, you'll find he puts on an excellent show.



Song: Life on Mars
Album: The Life Aquatic Studio Sessions
Site: SeuJorge.com

More hygiene hypothesis...

In the previous post, I mentioned what's most frequently called the "hygiene hypothesis," which considers the link between autoimmune disorders and antisceptic environments.

Is it possible our children are being raised in environments that are too clean and this is resulting in more asthma, allergies, etc. because underemployed immune systems start wandering around looking for things to attack inside the body?

Here's a link to a more recent Duke study that supports the idea.

On the way to work, it occurred to me this hypothesis provides a golden opportunity for avoiding housework:

"Honey, I'm sorry about the mess. I want you to know it bothers me just as much as you, but try to remember... it's for the kids."

Swingset Chain

Yet another aforementioned, fine mellow tune...



Song: Swingset Chain
Disc: It's Yours to Keep
Artist: Loquat
Myspace: Loquat

Pleasure, Pain and Rain

A couple years ago, the Edge asked scientists and intellectuals "What do you believe that you can't prove?" and I really enjoyed the pages and pages of big thinkers theorizing. In fact, half-baked theories are so much fun, I come up with a few of my own from time to time.

I recently sent a friend a bunch of musical links. When I asked him what he thought, he replied, "Man, everything was so mellow. You're gonna put me to sleep if you keep that up." We're roughly the same age, but now I'm left wondering if I've gone mellow and if so, why?

I've read a couple of articles about autoimmune disorders. Some researchers believe these problems are caused by raising children in environments that are too clean. Huh? According to theory, when our bodies aren't exposed to enough nasty stuff, our immune systems are left standing around with too little to do, so (being no slackers) they find something to do and attack us. Thanks, conscientious immune systems!

So what does this have to do with my friend saying my musical tastes are now too mellow? I think the key difference between us is kids. He doesn't have kids. I do. The half-baked theory? Lacking natural sources of screeching, banging and screaming, his subconscious leads him to seek manufactured sources of mayhem. Just like an immune system with not enough work. That's the ticket.

Here's something from the 80s with a little more edge than I usually post. From Love (still an excellent disc), The Cult singing Rain.

The Question of Smart...

A great analyst can understand a complex system by progressively breaking it into down into smaller and smaller parts. This is a reductive process requiring analytical skills and the ability to see what is and understand it.

A great inventor, on the other hand, can take a collection of small parts and build a useful complex system by assembling parts into greater and greater wholes. This is a creative process that not only requires an understanding of how all the parts work, but also the ability to see what is not and create it.

It seems to me that analytical skills and creative skills are distinctly different. They seem to be related but running in opposite directions in the same respect that deduction and induction are related but running in opposite directions.

They don't seem to necessarily go hand-in-hand. I've seen individuals with amazing analytical skills shrug in the face of great creativity and mutter, "I could never do that..." Likewise, who hasn't run into an amazing artist who is befuddled by mathematics and systems analysis?

Without turning this into a post about hiring engineers, I'd like to note that I believe the greatest engineers are blessed in both areas. There are really good engineers that can identify and fix the most formidable bugs; and there are really good engineers who can dream up perfect solutions; but a great engineer, I think, is one that excels both analytically and creatively. (If I had to guess, I'd say Apple understands this better than any other company creating software nowadays. Adobe seems to understand it pretty well too.)

Time after time, I've seen articles and blog posts proclaiming the importance of hiring people who are "smart," but I'm perpetually at a loss when it comes to defining what "smart" is.

People with great analytical skills often seem to equate "smart" with possessing great analytical skills. Likewise, people with great creative skills seem to equate "smart" with possessing great creative talent. All too often, "smart" seems to be a function of the particular cave in which one dwells. I share skepticism with Howard Gardner on this issue.

I've rambled on about this, because it's a question that has long intrigued me. Maybe I should add it to the slowly coalescing list of running themes in this blog.

Prompting me today is an article in BBC News titled Grey power: Battle of the Brains:

"But what about 'creativity'? It is not really tested by an IQ test.

Robert Sternberg, from Tufts University, Boston, maintains it's essential: 'Creativity was a tool for the high flyers - the Einsteins, the Darwins, the Newtons. But now the world has changed so much that creativity is now a vital part of intelligence for everyone'

We assessed creativity by using a test developed in the 60s: 'Name as many uses as you can for a sock in 10 minutes.'

The results were interesting. One of the high scorers on the IQ performed poorly; the other, extremely well. The creatives in our group also did very well, as you would expect."

Blogroll += Good Math

What's commonly referred to as Sturgeon's Law states "90% of everything is crud." With the advent of digital media, I'm afraid this has become an understatement, and if this blog serves any purpose at all, I hope it serves as a gateway for someone to something good.

This time, the recommendation is Mark Chu-Carroll's excellent blog, Good Math, Bad Math. His wisdom and lucid explanations of math and computer science are well done.

Shock Jock Algorithm

Shock Jock Algorithm in C Major:

while (shock < too_shocking)
   shock++, ratings++, money++;

exit(0);

Willy Mason

Mentioned before, this is Willy Mason singing Oxygen.



For more, check out the video for Save Myself and his Myspace page.

Official site: WillyMason.com

Vonnegut

Kurt Vonnegut. 1922 - 2007.

So it goes.
Rest in peace.
You'll be missed.

Birds & Stones

Bird #1. Some things are best explained with interactive examples. Recently, I ran into a need for one here and that led me to plunk around some more with JavaScript and ExplorerCanvas. Flash... yes, I know. :)

Bird #2. I've been trying to come up with a simple and fun way of introducing my kids to programming, but every candidate idea I've had so far has promised to be a little too much like drinking from the firehose.

Stone. I put together a simple little vector graphics laboratory on a web page. A code editor, a Run button and an output window all on one page! :)

As an intro, I told my son to change some numbers in the code to see what happens. (Naturally, the first thing he changed was the number of loop iterations to a ten digit number that locked up the page.)



Here's a link to version 0.1 of this project. The context parameter passed in is the rendering context. Here's Canvas tutorial on a Mozilla dev page.

Nerina Pallot - Sophia

Been busy with Easter gatherings and work.

Nerina Pallot... There are a number of comparisons coming to mind, but I won't make them, because I think they'll only become distractions...

This song is Sophia from her album Fires...

Sakamoto - Rain

After spending an inordinate amount of time looking for praiseworthy new music, I decided to throw in the towel. Sure there's great new stuff out there somewhere. Shrug. My answer: Sakamoto, 1987.

I don't know when or where, but this is a live performance of Rain (I Want a Divorce) by Ryiuchi Sakamoto, from the soundtrack of one of my favorite films, Bertolucci's 1987 film The Last Emperor. (IMHO, it's some of Sakamoto's finest work.)

A piano plays a relentless pounding stacato as a violin climbs one punctuated step at a time. Eventually everything breaks into a melancholic interlude, ebb and flow, until rain returns.

Clinks - C

As a follow-up to the Wired piece Mixed Feelings, check out the Tongue Vision blog.

Ebert, sorely missed, writes "40 years is not enough."

Although these are titled Very Difficult Analytical Puzzles, my guess is we bit-twiddlers have an edge.

What if a software pirate tried to turn himself in and nobody cared? (YouTube)

The first study I've seen on the possible effects of global warming on U.S. agricultural production (25% to 80% reduction?) (Statistical Modeling...)

National Geographic has an article on Toxoplasma gondii and rat mind control (link). If you're interested in more, Carl Zimmer has devoted a lot of time and energy to the subject (his excellent blog, The Loom).

Reach! A Lecture Musical Prank! (YouTube)

Yellow

Of Mice & Metamerism...

"Although mice, like most mammals, typically view the world with a limited color palette—similar to what some people with red-green color blindness see—scientists have now transformed their vision by introducing a single human gene into a mouse chromosome. The human gene codes for a light sensor that mice do not normally possess, and its insertion allowed the mice to distinguish colors as never before...

The new abilities of the genetically engineered mice indicate that the mammalian brain possesses a flexibility that permits a nearly instantaneous upgrade in the complexity of color vision, say the study's senior authors, Gerald Jacobs and Jeremy Nathans."

link

(via neurodudes)

Mixed Feelings

Yet another fascinating story of neuroplasticity.

Mixed Feelings by Sunny Bains in the latest WIRED.

"I cranked up the voltage of the electric shocks to my tongue. It didn't feel bad, actually — like licking the leads on a really weak 9-volt battery. Arnoldussen handed me a long white foam cylinder and spun my chair toward a large black rectangle painted on the wall. 'Move the foam against the black to see how it feels,' she said.

I could see it. Feel it. Whatever — I could tell where the foam was. With Arnold ussen behind me carrying the laptop, I walked around the Wicab offices. I managed to avoid most walls and desks, scanning my head from side to side slowly to give myself a wider field of view, like radar. Thinking back on it, I don't remember the feeling of the electrodes on my tongue at all during my walkabout. What I remember are pictures: high-contrast images of cubicle walls and office doors, as though I'd seen them with my eyes. Tyler's group hasn't done the brain imaging studies to figure out why this is so — they don't know whether my visual cortex was processing the information from my tongue or whether some other region was doing the work."

link

X'X inverse X' and SVD

In the process of knowledge gentrification, I continue to meander my way back and forth through issues of linear algebra...

This is post #6 in a series discussing the expression (X'X)^-1X'Y used in linear regression. Previous posts are here: #1, #2, #3, #4, #5.

Most of the literature I've read on least squares discusses the orthogonal projection of Y onto the column space of X, but something I haven't seen mentioned much is that (X'X)^-1X' is the Moore-Penrose pseudoinverse of X.

X⁺ = (X'X)^-1X'

Now's a good time to bring the SVD back into the picture. (Actually, now that I've made it this far, I think I need to go back and amend an old post a little bit.)

Given the SVD of X as USV', consider (X'X):

(X'X) = (VS'U')(USV') = VS'U'USV' = VS'SV'

And:

(X'X')^-1X' = (VS'SV')^-1(VSU') =

(VS^-1S'^-1V') (VSU') =

VS^-1S'^-1V'VSU' =

VS^-1S'^-1SU' =

V(S'S)^-1SU' =

And since (S'S)^-1S = S⁺:

VS⁺U' = (X'X)^-1X'

This shows the equivalence between the SVD pseudoinverse and (X'X)-1X' and why the SVD pseudoinverse gives the least squares solution. (I think this applies when the number of columns is less than the number of rows. Need a reference for that.)

(Note: V and U are orthogonal matrices, so their transposes and inverses are one and the same. S is a diagonal matrix, so it's equal to its own transpose. The transpose of a product is equal to the reverse product of individual transposes. Likewise, the inverse of a product is equal to the inverse of the individual products.)

The Namesake

Mira Nair (Monsoon Wedding, Vanity Fair) directed this adaptation of Jhumpa Lahiri's novel.

Newsweek's David Ansen hit a nail on the head: "There are times when you wish the movie was a mini-series. This is meant both as a tribute, for the Ganguli family is so engaging you'd be happy spending much more time with them, and an acknowledgment that a tale this expansive doesn't always fit comfortably within the constraints of a feature-length frame."

At times it seemed a little discontinuous, but it's one of those films that leaves you feeling you should call your parents, wondering what sort of impression you'll leave with your children, ruing the brevity of life. Films that manage to do that are worth seeing.

Pug Bowling and Patents

I'm not an intellectual property lawyer, so don't take my words as the gospel, but I began pondering the question of pug bowling somehow infringing on U.S. Patent 5,443,036. My guess is it doesn't, since the claims of 5,443,036 are explicit in references to felines:

"A method for inducing cats to exercise consists of directing a beam of invisible light produced by a hand-held laser apparatus onto the floor or wall or other opaque surface in the vicinity of the cat, then moving the laser so as to cause the bright pattern of light to move in an irregular way fascinating to cats, and to any other animal with a chase instinct. "

Also, the point of pug bowling doesn't seem to be wholly one of exercising the pugs. The more serious players are probably in it for the love of the game.

That said, my guess is any attempt to patent pug bowling should probably cite 5,443,036 as prior art.