Hothothothothothothothot

http://www.google.org/publicalerts/alert?aid=64e887f3f3677268&hl=en&gl=us&source=web

You know what that link is? A heat alert for the Montreal Region. It has been between 80 and 90F (26 and 32 C). That may not sound like much, but the humidity is high and a storm that has been supposed to break just keeps hanging out in the sky. Also, houses here don't really have air conditioning, so I am continuously covered in a sheen of sweat. I've been taking cold showers in the evening as an attempt to cool my body down to a sleeping temperature, but it doesn't last me through the night. This morning I had to take a second cold shower at 5am to get back to sleep.  I think I may start sleeping at the lab if this keeps up because the lab is cold, if not particularly comfortable.

Research is hard

Before I start on actual content: I changed my blog template! I had had the original one since approximately January 12, 2008. That's 6 years ago. This one is kind of boring, but the gray background won't mess up pictures too much.

But anyways, research is hard. I'm trying to making a neural net that will read in a bunch of series of words (called ngrams) and try to use the middle word to predict the words around it. You might say, "that doesn't sound very useful, why would you want to predict words around a word?" and I totally agree; it isn't very useful. What is useful, is that to predict the surrounding words, the system first projects the middle word into a vector in a continuous space (at first to a random spot). Then, when it predicts wrong, it adjusts the vector a little bit in the direction that would have predicted the right word. After many iterations, these vectors end up containing information about the word that is useful to the system when predicting nearby words. Since similar words tend to be surrounded by the same words, similar words get clumped together. Once we have these projections, we can do cool stuff like translate phrases from English to French semantically rather than just using the most common words.

So I write the code for this neural node, run it on a small piece of data, and it seems to learn (the number it gets wrong while training goes down). Cool beans. I get the actual data and holy moley, there are 350,000,000 samples inside. My code can run about ~225 samples per second, so that would take about 3 weeks... Right now I am trying to run 10,000,000 samples, and it is not going so well. Not only is it taking longer than the predicted 12 hours, but after going down at first, the cost is now wavering back and forth very far from 0. This probably means my learning rate is too high, but the cost does seem to be decreasing slowly if I look at it from afar. 

To make us all feel better, here are some pictures from my weekend wandering:

A bakery with no sugar and white flour. It has integral bread, but that just sounds derivative.

Don't cook, just eat!

This picture isn't funny, but that bread looks delicious.

I too drink my mushrooms as tea.

I'm in Canada!

After many months of stressing about whether I was going to be allowed to work in Canada, I successfully got my work permit and started my internship at the beginning of this week. Yay! I'm doing computer science research in Montreal, which is pretty darn awesome. My coworkers are a fun bunch: to celebrate someone's Doctorate, one student got him a machete, and we hiked up the mountain to play real life fruit ninja! Turns out coconuts are really hard to slice, but melons work surprisingly well.

Montreal is an interesting city. In the US, I generally know what language to use when talking to people: if they are my parent's French friends, I use French, otherwise I use English. In Montreal, most people speak both, but would be more comfortable using one language over the other. It's a confusing guessing game.

Nonetheless, I have definitely been enjoying life here. I love walking around my neighborhood and checking out the little grocery stores and cafes. I seem to be living at the intersection of a Greek neighborhood and a Hasidic neighborhood, so there are a lot of Greek restaurants and kosher grocery stores. Even the generic grocery store has interesting greek cheeses that I had never tried before (including Mizithra, which my Greek friend keeps talking about). This morning I also saw freshly grilled Souvlaki being sold outside one of the Greek restaurants! I'll have to try that next week (or perhaps tomorrow if it is still there).

I've been lazy about bringing my camera places, but I promise I'll be better next week. In the meantime, here are some things I've found with my phone camera:

There is a fruit store near my lab that has some nice looking and rather inexpensive fruit. It's "La Sucrerie" and it sells fruit, pastries, and maple syrup. It also does not seem to exist on the internet (...?) Unfortunately, it is really far from my apartment, so I won't be going shopping there very often.

There is a used bookstore about 5m from my apartment building! It does not seem to have much Sci-Fi/Fantasy, but as I only brought 2 books with me, I'll probably end up getting some books from there :)

As a final treat, here is a video of the post-doc I'm working with slicing through a cantaloupe :)