a page of tumbling madness

Explorations in Clojure's core.logic

Wed, 20 Feb 2013 10:06:00 -0800

I first heard of core.logic three or four months ago, and I’ve been trying to learn more about it ever since. It’s apparently an implementation of miniKanren in Clojure, where miniKanren is a small language implemented in Scheme that’s a stripped-down version of a system called KANREN.
Long story short, core.logic is a library that lets you do logic programming in Clojure.

Okay, so what’s logic programming?

I’d never heard of logic programming before, probably because I didn’t pay as much attention in school as I should have. Wikipedia says that it’s one of the four main programming paradigms, the other three being object-oriented, imperative, and functional. Here’s a simple logic program:

https://gist.github.com/jrheard/76d49b27516df0cdf077

Let’s look at this program one step at a time. We’re invoking run*, which is a macro that gives you a fresh logic variable (which we’ve bound to the name q), lets you specify a series of goals that might relate to that logic variable, and returns a list containing all of the values that the logic variable can possibly take on in order for those goals to be fulfilled.

Here’s another simple program:

https://gist.github.com/jrheard/357fa4576d01e5601835

The result of this program is a list containing one value, _0. _0 denotes a fresh logic variable; the fact that our program returns _0 means that q can take on any value at all and the goals in the body of the run* call will still hold true. Neat!

At this point, I’d like to refer you to Ryan Senior’s post “The Magical Island of Kanren”. I like the metaphor he uses in that post; he also has some great examples, and he introduces conde, which we’ll be using in a second. You’ll need to have read and understood that post in order to make sense of the rest of this one. It’s short and simple, go read it and come right back!

DOM-logic

Okay: now that we have the beginnings of an understanding of core.logic’s vocabulary, let’s get to the reason I wrote this post.

core.logic allows you to ask questions about your data in a declarative way. It’s kind of like SQL in that respect, but it’s embedded in Clojure (so you have all of your favorite functional-programming tools available) and you don’t need to send all of your data over to a database before you can ask questions about it. Ryan Senior’s “Practical core.logic” talk (particularly minutes 7 through 13) really helped crystallize that for me, and right after watching it, I thought to myself - wouldn’t it be fun to write a simple core.logic program that implements document.getElementsByClassName? So I did. Let’s look at it!

To get started, we’ll need to import some libraries, pull down a Web page, and parse its HTML into a DOM-tree representation. Here’s some code that does that:

https://gist.github.com/jrheard/d854c12384108ef12a3e

Now that we’ve got our data, let’s start writing our logic program. core.logic programs are expressed using relations, which is another way of saying “functions that return goals”. We’ll need to write a relation that takes a node and a classname and then returns a goal that says that the node should have the specified classname. Here’s one:

https://gist.github.com/jrheard/3c09a19e9a99947aad63

We introduce a relation called secondo, and use it to bind a fresh logic variable named attrs to the second element of the node, since the second element of a node is a map containing the node’s HTML attributes. Once we’ve done that, we can say that attrs should contain a k/v pair like e.g. {:class "user-passport"}. Goal written!

Now that we’ve got a way of saying that a logic variable that represents an HTML node should have a certain class name, we’re going to have to give ourselves a way of applying that goal to every element of the DOM tree. Here’s what that looks like:

https://gist.github.com/jrheard/16269f93dfc51d511792

One step at a time: to start off, we introduce a relation called childreno. It takes two logic variables, one representing an HTML node, and one representing the node’s children, and then uses resto twice to unify children with the (potentially empty) list of all elements in node after the first two elements, because that’s where the node stores its children.

Next up is nodes-existo, which is the workhorse of this program. This relation takes a relation and a couple of logic variables. It starts by using conso to split the passed-in nodes lvar apart into node and remaining-nodes. Now we use a conde to say that we want our program to check each of three cases. (And when I say “each”, I mean “each and every!” Note that while, as pg says, a cond expression will evaluate “only an L-shaped path of its subexpressions”, a conde doesn’t short-circuit like that - once it’s done checking one of its lines, it’ll backtrack out and check the next, regardless of whether the first line’s goals succeeded or failed.)

The first line’s simple enough: we say that if the goal returned by a call to (condition node) succeeds, then node should be unified with out.

In the next line, we introduce an lvar called children, use childreno to unify it with node’s children, and recur on those children.

In the third line, we recur on remaining-nodes, which we used conso earlier to unify with the rest of the nodes in nodes.

And that’s basically it! All that’s left is get-elements-by-class-name, which is a plain old function that takes a node and a classname and uses nodes-existo and has-classnameo to tell run* how to identify the possible values of q. Simple!

So, what has this bought us?

At this point, the program we’ve written doesn’t look that much different from how we would have written it without core.logic. What’s the logic-programming approach bought us so far?

Well, for starters, we haven’t had to worry at all about assembling the list of nodes that we want to return. All we do is tell our program how to identify the sort of values we’re interested in, and tell it where it can look to potentially find more of them, and run* handles the rest. So that’s cool! One other cool property about this program is that you can run it backwards trivially - you can give it a classname and one or more nodes with that classname, and it’ll generate all of the possible DOM trees that could contain that node; but that’s not exactly useful in this particular situation.

So at this point, I’d say that core.logic has let us write our program slightly more declaratively, but this program probably isn’t a lot shorter than the non-core.logic version of it would have been.

Enter matche

I noticed in Ryan Senior’s talk - as well as in his article on appendo the great - that he used a construct called matche, which brings the joy of pattern matching to your core.logic programs. As far as I can tell, it’s only fully documented in Appendix C of William Byrd’s thesis, but Ambrose Bonnaire-Sergeant has a good introduction to it near the end of his great core.logic intro. Let’s take it out for a spin!

As long as we’re making changes, let’s first make has-classnameo a little more general - let’s rename it to has-attrso, and have it take a map of attributes (e.g. {:class "foo" :id "bar"}) that we want matching nodes to have. We can define it using defne, which is just shorthand for defining a function that does a matche on its arguments:

https://gist.github.com/jrheard/c5a6e8919995e985ffa2

Note that we’re able to completely throw secondo away - we just say: this is what a node looks like, and here is a constraint that we want to apply to the node’s attributes. Neat! Now let’s take a look at the redefinition of big bad nodes-existo:

https://gist.github.com/jrheard/ad1027a4b2f8fcdef54c

We’re able to inline the opening conso and fresh calls into the implied matche expression, which is cool. The first two lines of the conde look pretty familiar, but the line concerning the node’s children has changed a lot. We’re now using another matche in order to pull ?node apart and get at its ?children. This saves us a fresh call, a childreno call, and actually lets us delete the definition of childreno entirely.

Here’s our final program in its entirety:

https://gist.github.com/jrheard/f1d3fc8b7e4d45e5a0f5

This is about as declarative as it gets, folks. Our program grabs some data, and then very simply and succinctly (the meat of the program is ten lines of code!) tells run*: this is what the nodes we’re interested in look like, and this is where the rest of the nodes you’ll want to look at are; please find the good ones for us. Logic programming and pattern matching are a truly fearsome combination. I’m really excited about core.logic - it seems like a really useful tool.

Notes

Here’s an earlier version of nodes-existo:

https://gist.github.com/jrheard/a0d103dbb4f64f51aecd

It has a subtle bug, because out of reflex, I thought I had to define a base case. Can you find it? How do you think the bug will change the behavior of the program? Do you understand why the code works when that line is removed?

Also: our code to get elements by class name won’t work on elements that have multiple class names - that is, if we use this code to search for elements that have the class "foo", elements that have the classes "foo bar" won’t be matched. Enhancing this code to support that behavior is left as an exercise for the reader.

Resources

If you want to start diving into this stuff, here’s how I’d recommend doing it:

Read the official core.logic primer.
Watch this video by the miniKanren guys. It has what I seriously think is the coolest demo of anything ever. You’ll know what I’m talking about when you see it.
Watch this other video by the same guys.
Here’s another one - it shares a lot of content with the first two videos, but in the final fifteen minutes they build and demo a working type inferencer from scratch for the hell of it. It’s really a treat to see these guys at work.
Read The Reasoned Schemer.

That’s actually about as far as I’ve gotten so far - I bought a copy of “The Art of Prolog” that I’ll read at some point, and I’m working on getting a hard copy of Byrd’s thesis to read through. I also have core.logic’s source code open in a tab and will hopefully read it all one of these days, and the author of core.logic has a recommended reading list. Aside from that, though, I’m not aware of many other resources besides the ones I cited throughout this post - this field is the coolest thing I’ve ever heard of, but it seems relatively small as of yet. Hopefully that’ll change as more and more people are introduced to it via core.logic!

(Acknowledgements: I’d like to thank the miniKanren guys and David Nolen for existing, everyone whose articles and documentation I cited for writing those things, and my roommates’ cat Isis for sitting on me for the entire five hours I spent writing this post.)

getting started with clojure

Tue, 08 Jan 2013 10:27:00 -0800

I’m about to try to teach a bunch of people (primarily Python devs running OS X) how to use Clojure, and I’m not satisfied with any of the currently existing documentation on how to get up and running from scratch. When I was going through all this myself a few months back, there was a weird period of a good few weeks when I had basically no mental map of the Clojure ecosystem and had no idea how to assemble one.

My goal for this post is to create the resource I wish I had six months ago. I’ll assume that you’re running on OS X and have a non-zero amount of programming experience.

The Clojure Book

Your first step should be to buy and begin reading Clojure Programming. There’s another book called (confusingly enough) “Programming Clojure”, and I can’t vouch for whether it’s better or worse, but I used “Clojure Programming” and liked it very much, so it’s what I recommend. It’s written by people whose names you’re going to get used to seeing everywhere as you explore the Clojure ecosystem; all the main figures in the Clojure community seem to be inhumanly prolific.

Let’s Get Started

Now, let’s start getting your environment assembled. Get Homebrew - “the missing package manager for OS X” - if you don’t have it already, and then run

brew install leiningen

Congratulations, now you have Leiningen! (Make sure you ended up with version 2.0 or greater - you can check that by running lein --version.)

So what the hell is Leiningen?

Leiningen’s the main tool you’ll be using for:

starting up a REPL
downloading+installing libraries
running your programs
starting a server to run the webapps you’ve written

Go ahead and run `lein repl`. You’ve now got a working Clojure REPL! In addition, if you run that command from the top-level directory of one of your Clojure projects, it’ll deal with wiring up classpaths and whatnot so that you’ll be able to import and play around with your project’s code and the libraries that it depends on. We’ll get to that later. Right now, let’s create a skeleton project for us to play around with by running

lein new foo

When that’s done, cd into foo and you’ll see that it’s already got some files and directories:

[jrheard@jrheard-air:~/dev/foo] $ ll
total 16
-rw-r--r--  1 jrheard  staff   193B Jan  5 15:17 README.md
-rw-r--r--  1 jrheard  staff   263B Jan  5 15:17 project.clj
drwxr-xr-x  3 jrheard  staff   102B Jan  5 15:17 src
drwxr-xr-x  3 jrheard  staff   102B Jan  5 15:17 test

Whenever you write a Clojure library/program/anything, your source code will live in the “src” directory and your tests will live in the “test” directory. Straightforward enough. Let’s take a look around in src:

[jrheard@jrheard-air:~/dev/foo] $ cat src/foo/core.clj

(ns foo.core)

(defn foo
  "I don't do a whole lot."
  [x]
  (println x "Hello, World!"))

Looks like Leiningen’s already created a file called “src/foo/core.clj”. It’s a Clojure program that defines a namespace called “foo.core” and then declares that that namespace contains a function called “foo”. Let’s check it out. Start up a repl with `lein repl` and poke around. Remember when I mentioned earlier that leiningen takes care of setting up your classpath and associated goop such that you’re able to access your project’s code from the REPL? Check this out:

user=> (use 'foo.core)
nil
user=> foo
#<core$foo foo.core$foo@6ad591a6>
user=> (foo "jrheard")
jrheard Hello, World!
nil

Awesome - we were able to import our code and run it. The `use` function basically serves the same purpose as `from foo.core import *` would in Python, and its use in source code is similarly discouraged for the same reasons that import * is discouraged. Like import *, though, It’s pretty useful to have when you’re poking around in the REPL.

So that’s cool - we’ve created a project, it’s got code in it, we’ve found out how to start up a working REPL that can play around with that code. Bullet point 1: accomplished. Let’s take a look at the second bullet point:

Downloading and installing libraries

You’re probably used to getting your libraries by running something from the command-line,
e.g. `pip install this_great_library_i_found`, which would download the specified library and install it either globally or within your current virtualenv. Things work a little bit differently in Clojure.

First, you’ve got to find a library that looks useful. The Clojure Toolbox is a fantastic tool for this, and is the best such resource I’ve found. Let’s choose a library to play around with: making HTTP requests is fun - let’s go down to the “HTTP Clients” section and see what our options are. Looks like we’ve got to pick between clj-http and http.async.client - but how do we choose?

Currently, my favorite way of deciding between competing libraries is: pull up their respective github repos, compare the number of stars+forks, and give bonus points to any libraries that have commits from within the past month or two. Not exactly scientific, but it’s served me well so far as a good proxy for the strength of the library’s community/influence/adoption. As of this writing, clj-http has 242 stars to http.async.client’s 127, so let’s pick clj-http.

So… how do we get it?

Let’s go to clj-http’s github repo. Check out how the README’s installation section has this block of code:

[clj-http "0.6.3"]

That’s the information we need - it’s a Clojure vector with two items, the first of which is the name of the library, and the second of which identifies the most up-to-date stable version available. We’re going to add this to our project.clj, which you saw earlier when we looked at the contents of the ‘foo’ directory. Open up project.clj, it’ll look like this:

(defproject foo "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.4.0"]])

Note the :dependencies section - it’s a Clojure vector containing one item, and that item is itself a Clojure vector containing two items. This vector indicates to Leiningen that we want our project to run on version 1.4.0 of Clojure. Fair enough - now let’s add the clj-http vector we saw earlier. Our project.clj should now look like this:

(defproject foo "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.4.0"]
                 [clj-http "0.6.3"]])

And that’s it! We’ve now specified to Leiningen that we want the clj-http library, and which version we need. Let’s try it out - start a REPL with `lein repl`, and let’s play around with our fancy new library. Notice that Leiningen will first download clj-http before starting up the REPL - that’s because it first runs `lein deps` behind the scenes any time you ask it to do basically anything, and that causes it to scan your project.clj and make sure that it’s already fetched all the dependencies you’ve asked it to.

Okay, back to our REPL session. Looks like clj-http’s github repo’s README suggests that you require it in the REPL by running

(require '[clj-http.client :as client])

So let’s do that - it’s the same thing as `from clj.http import client` in Python (as opposed to `from clj.http.client import *`, which is again what the `use` function does.)

user=> (require '[clj-http.client :as client])
nil
user=> (client/get "http://www.yelp.com")
;; a big huge blob of data pops out!

Okay, wow, looks like that worked! That’s sort of hard to read - you’ll notice that the big huge blob of data ends with a “}”, which is a hint that it might be a Clojure map. Let’s try poking at it:

user=> (def resp (client/get "http://www.yelp.com"))
#'user/resp
user=> (type resp)
clojure.lang.PersistentArrayMap
user=> (keys resp)
(:cookies :trace-redirects :request-time :status :headers :body)
user=> (:status resp)
200
user=> (:headers resp)
{"server" "Apache", "content-encoding" "gzip", "x-proxied" "lb2",
"content-type" "text/html; charset=UTF-8", "date" "Sun, 06 Jan 2013 00:02:58 GMT",
"cache-control" "private", "vary" "Accept-Encoding,User-Agent",
"transfer-encoding" "chunked", "x-node" "wsgi, web40, www_all",
"x-mode" "ro", "connection" "close"}

And there you have it - we’ve found an HTTP client library, downloaded it, and figured out how to use it interactively in the REPL!

It took me a while to figure this all out - after beating my head against a wall for a day, I eventually had to jump into the #clojure IRC channel and plead for help. Now you don’t have to! For further reading, check out the official Leiningen tutorial.

Putting it all together

Let’s finish up by figuring out how to actually run a Clojure program. Let’s try a good old `lein run`:

[jrheard@jrheard-air:~/dev/foo] $ lein run
No :main namespace specified in project.clj.

Okay, that didn’t work. Referring back to the Leiningen tutorial mentioned earlier and doing a search for :main, we see that you can define a :main key in your project.clj definition that specifies the namespace that `lein run` will run, and that said namespace has to contain a `-main` function, which serves as the entry point into your program. Let’s add this line to our project.clj spec:

:main foo.core

And finally let’s modify src/foo/core.clj so that it looks like this:

(ns foo.core
  (:require [clj-http.client :as client]))

(defn -main
  "Prints the first 50 characters of the HTML source of yelp.com."
  [& args]
  (println (apply str
                  (take 50
                        (:body (client/get "http://www.yelp.com"))))))

Here we go - let’s try it out with `lein run`!

[jrheard@jrheard-air:~/dev/foo] $ lein run
Compiling foo.core
<!DOCTYPE HTML>

<!--[if lt IE 7 ]> <html xmlns:fb

It works!

That’s it for now - you now have a working REPL to play around with, the ability to install and use libraries, the knowledge to give your programs access to those libraries and run them, and a really good book that’ll take you through everything else you need to know about the Clojure language.

The reason I had to write this post

Clojure’s still a pretty young language. The community is extremely small relative to e.g. Python’s, and although the core language’s API is (I’m told) remarkably stable, a lot of the tools around it are new and in a state of rapid change. Add on top of that the fact that most of the up-to-date documentation you’ll find has poor SEO - to the degree that a lot of your Google searches will turn up documentation on richhickey.github.com that’s years out of date and deprecated - and you’ll find that getting started from scratch can be a little tricky.

I hope that this post has helped save you the few weeks of bewilderment that I went through when I was getting started - I promise that the joy of actually programming in Clojure is well worth putting up with these growing pains.

Assorted resources

Watch these two lectures by Rich Hickey, the creator of Clojure: “Are We There Yet?” and “Simple Made Easy”. In particular, watch the first one three or four times over the course of several months.
I’ve spent the past few weekends watching a whole lot of Clojure lectures. The official Clojure youtube channel is a great resource, and the amount of great content on InfoQ is really astounding, I’ve probably watched at least 25 lectures there.
The Clojure Toolbox, mentioned above.
Where Did Clojure.Contrib Go - you’re going to see references to libraries like “clojure.contrib.monads” as you explore. clojure.contrib no longer exists, and this page will tell you where the libraries it used to contain have gone.
This example project.clj shows you how to take advantage of the thousand different hooks Leiningen provides for customizing how your project is built and run.
Assorted core members of the Clojure community worth following on Twitter: @cgrand, @cemerick, @marick, @weavejester, @stuartsierra, @seancorfield, @Baranonsky, @richhickey
Read “The Joy of Clojure” once you’re done with “Clojure Programming”.
Heads up - Noir is deprecated. Use Compojure instead.

"It takes an awful lot of work to be the sixth most impressive thing Scoble (or the New York Times,..."

Wed, 16 Nov 2011 10:00:47 -0800

“It takes an awful lot of work to be the sixth most impressive thing Scoble (or the New York Times, or Techcrunch, or $PICK_A_GATEKEEPER) will see this week. It takes much, much less work to offer a better user experience than a) an Excel spreadsheet sent over email, b) having your office manager act as the endpoint to a REST API, or c) doing anything involving actual paper.”

- patio11

dropping all tables on postgres using flask-sqlalchemy

Sun, 13 Nov 2011 14:53:00 -0800

Archiving this so I don’t forget it: in Flask-SQLAlchemy on a Postgres DB, db.drop_all() can fail silently. This can be somewhat infuriating.

This link sums up the situation pretty well. There are a bunch of hacky fixes floating around Google, none of which work, except for a particularly simple one which Mike Bayer himself proposes in the comments of that same link:

db.reflect()
db.drop_all()

db.reflect() is the secret sauce that makes the drop operation work. Which makes sense, I guess.

The catch is that db.reflect() is 100% broken in the current version of Flask-SQLAlchemy. Hopefully that gets fixed soon. In any case, if anyone else is living in a world where db.drop_all() doesn’t drop any of their tables: call db.reflect() first.

How to add a free shared PostgreSQL db to your python app running on Heroku

Sun, 06 Nov 2011 12:25:00 -0800

Heroku’s documentation claims that “New apps created on Heroku automatically have a shared database installed.” Unfortunately, that isn’t at all true for Flask apps: if you create a new Flask app on Heroku, you don’t have any database at all by default.

I spent a couple of hours Googling and digging through documentation to try to figure out how to get a database for my app to use, and was finally helped out by user lukemelia in #heroku. If you find yourself in the position I was in, here’s all you need to do:

(moody) [jrheard@jrheard:~/envs/moody] (master) $ heroku addons:add shared-database:5mb
-----> Adding shared-database:5mb to moody... done, v11 (free)

Problem solved.

a personal fitbit dashboard

Thu, 03 Nov 2011 09:47:00 -0700

I bought a Fitbit in July, and ever since, I’ve been walking a lot. I walk home from work most nights, and on the weekends I like to walk to the ocean. I’ve been trying to walk fifty miles a week for the past few months, and I’ve hit that goal most of those weeks.

I’m doing mainly because the Fitbit universe is made of progress bars, and if you put a progress bar in front of me, I will walk on hot coals if it’ll make the thing move a little bit to the right.

This is a snippet of what the main Fitbit web interface looks like:

You’ll note that the dang thing is covered in progress bars. I’m helpless here.

The most important progress bar is the one in the top right: the weekly mileage goal. When I first got the gadget, I sort of just started walking a lot, and found that I could hit 35 miles per week without too much effort. From there, I just kept bumping up the goal until it actually took a little bit of work to reach, and now it’s set to 50. Also, 50’s a nice round number, so that was probably a factor too.

The thing is, since the weekly progress bar resets at midnight on Monday morning, if I don’t remember to check the thing before I go to bed on Sunday, I’ll have no idea whether or not I hit that weekly goal. That’s where the weekly Fitbit progress report email comes in:

So that’s pretty cool: every Monday afternoon, I get a pretty email that tells me whether or not I hit my weekly goal. Hunky-dory. In fact, I really much prefer the weekly email to the current Fitbit dashboard. Look at the font size on those numbers!

After a few weeks, I found myself wishing for a Fitbit web dashboard that was a lot more like the weekly email, with big numbers, a focus on how much you’ve accomplished so far this week, and what you’d need to do for the rest of the week in order to meet your goal.

So I built one. It’s really simple.

I have a cron job that updates this data every few hours, and always leave a tab open on this page so I can see where I’m at at any given point during the week. Take a look at what it looks like midway through the week, when I’ve still got lots of work left to do, or on a Sunday of a particularly productive week.

Code Notes

If you’re a Fitbit user and want to use this thing yourself, feel free: the full source code for this dashboard is available at github.com/jrheard/fitbit. The two files where the meat of the work is done are fitbit.py and index.tmpl; I wrote this as a one-off side project in a weekend, so this isn’t exactly production code - prepare yourself for <table> tags and a constant named DISTANCE_TO_PORTLAND.

When I made this thing, I was under the impression that Fitbit didn’t have a web API, so I approximated one by just scraping a few logged-in pages. On further reading, it looks like they do in fact have an API that would have probably given me the information I needed, but it’s sort of unintuitively named and didn’t jump out at me when I was browsing their developer site. Oh well!

This was my first time playing with the Mako templating engine. Overall, I was pretty pleased with how simple it was: I like the general flow of “come up with a dict of key/value pairs, feed it to the template, there is no third step.” Simple is good.

This was also my first time using the python Requests library, which proved to be pretty indispensable. Its session object let me log in on fitbit.com and then stay logged in while I made requests to other pages in order to retrieve additional information, which would have been a pain to do with urllib. The relevant code looks something like this:

import requests

with requests.session() as session:
  # log in
  session.post('https://fitbit.com/login', data={'username': 'foo', 'password': 'bar'})

  # remain logged in by continuing to use the same session object to make further requests
  response = session.get('http://www.fitbit.com/my_personal_info')
  print response.content

PyQuery came in handy as always - I think I’ve used it in at least half of my random side projects at this point. Beats the hell out of BeautifulSoup.

That’s about it for this post. Major props to the Fitbit team for building a neat product that’s keeping me healthy and giving me an excuse to explore the city I live in. Hit me up on twitter at @jrheard if you’ve got any suggestions for a killer feature I left out of the dashboard.

"Another thing, very important for problem solving, is asking my colleagues, “How would you..."

Sat, 15 Oct 2011 12:06:31 -0700

“

Another thing, very important for problem solving, is asking my colleagues, “How would you solve this?”

It happens so many times that you go to them and you say, “I’ve been wondering about whether I should do it this way or that way. I’ve got to choose between A and B,” and you describe A and B to them and then halfway through that you go, “Yeah, B. Thank you, thank you very much.”

”

- Joe Armstrong, inventor of Erlang

"By and large, writing code is more essay than puzzle."

Wed, 16 Mar 2011 14:50:42 -0700

“By and large, writing code is more essay than puzzle.”

- http://duartes.org/gustavo/blog/post/lucky-to-be-a-programmer

Photo

Wed, 29 Dec 2010 22:43:34 -0800

me: just got into work, just glanced at the last couple of lines i typed before leaving last night

Tue, 09 Nov 2010 10:27:57 -0800

me: just got into work, just glanced at the last couple of lines i typed before leaving last night
me: [jrheard@dev09:~/pg/loc] (category_ordering) $ grep -ir f[u+]ck *
me: ai/trie/ctrie.pyx: # "If your strings are longer than 1024, why the fuck are you using a trie?"
me: ctrl-C'd before it got farther than that, evidently
pepper: haha
pepper: ha
pepper: haha
pepper: if you did that to the microsoft word code, i put a "gimme a fucking console biotch" in there somewhere

know thyself

Tue, 18 May 2010 22:20:00 -0700

i’ve been working on a completely friggin’ awesome project with a couple of really smart guys for the past couple of months. as agreed, i split off from the development of the primary aspect of the project a couple of weeks ago in order to focus exclusively on creating its social-network-esque website counterpart, with my teammates continuing to work on preparing the main part of the project for its upcoming 1.0 release.

so, as a result, i’ve been working as the sole committer on a separate (and private, thankyouverymuch) github repo for the past couple of weeks. as it turns out, one neat feature that github has is the ability to show you really cool graphs about your repository, among them the “punchcard” graph, which plots your project’s commit activity over a period of time.

that’s me up there. those are my sleeping habits, my working habits, my partying habits, over the course of two weeks. the bigger the dots are, the more i worked at that intersection of day and hour.

that is so wild.

equally wild is the fact that the main project - the one that jhurwich and dmacdougall are still hard at work on finishing up (and which my buddy jackie is busily designing a few icons for) - has eerie, unintended parallels with this Punchcard graph thing: it’ll also tell you a whole lot that you may not have known about yourself and your habits, albeit in a completely different way. even though that wasn’t really what we had intended. at all. more on that in a couple of weeks.

seven more albums of awesome coding/studying music

Wed, 17 Mar 2010 21:09:19 -0700

http://jrheard.com/music/mice_parade.zip

http://www.last.fm/music/Mice+Parade

bem-vinda vontade’s my favorite.

snippets from the craigslist computer gigs RSS feed

Thu, 11 Mar 2010 10:00:00 -0800

In return for assistance, you will receive the following:

* improved resume

I am looking for the best of the best with JAVA skills. I need Architects, Engineers, BI/Data Modeling guys, UI and even predictive engine engineer. This is a big deal. I dont think this level of brain will be looking on here…. but I do think that some of us who know these elite top 1%’ers might be on here. SO, if you can refer me to a genius, and it results in a hire I will give you a $1,000 referral fee. It is that simple.

I have an outdated website and am looking to have a new one built. I am offering trade for yoga therapy/ and or therapeutic massage therapy. I would like to find someone who is a yogi and or marketing expert who at least understands the industry. […] Afterall isnt that the point is for people to find us on the web? I am clear about what I would like to have, this includes a you tube video of my work on the website, a blog, and equipped to be found on twitter, facebook, a schedule,etc. I have new artwork of a logo I would like to have built in so this would be helpful if someone understands graphics and how to get the art work into digital form that can be used as a template for the website or used in the website somehow.

We need and intern to help with a quick / powerpoint / Apple Keynote Presentation.

This is an unpaid intern gig for a new media group. (Lots of people we tried just did not have the skills.

But if your presentation rocks we will hire you on a project basis for the next gig and we have plenty in the pipeline!!

Please send resume with a cover letter imbeded in the response. No cover letter
(Their is a dude from Mumbai that just sends resumes to everyone and the SPAM is killing us._

We would love to be your first client and would gladly advertise your services!

on functional programming

Mon, 08 Mar 2010 08:00:00 -0800

an eye-opening comment

Our programming model does not have to match how our computer works.

I think one of the biggest intellectual hurdles with functional programming is that we know, deep down, that modern computers don’t work like that. They do have side-effects and changing, carried state. It’s a revelation to realize that the programming model you use to write programs doesn’t have to match how the computations will be physically carried out.

- HN user scott_s in response to Ask HN: Functional Programming Differences

this is a concept i’m still coming to terms with.

a few weeks ago, brian carper wrote an article in which he detailed the process he was going through while writing a final-fantasy-style RPG in clojure, from scratch, despite his oft-repeated assertion that he had “no idea what i’m doing.” one of the things about his writeup that came pretty close to blowing my mind was the way in which the game world was handled in functional-programming-land.

I’m sometimes amazed how far I can get before I need mutable state at all. The vast majority of my functions take a world value (a plain old hash-map) as an argument, and return a new world value after making changes to it. The current state of the world is whatever value is currently in the global WORLD ref.

The render loop grabs a snapshot of the world from the ref on each iteration, and then draws it. Thanks to Clojure refs, the snapshot of the world is guaranteed to be consistent (e.g. no NPC objects in the middle of mutating themselves) and persistent (the world value sticks around as long as the renderer needs it, even if the WORLD ref is changing in another thread). Once it’s been drawn, the renderer throws the world snapshot away and it’s garbage-collected later.

This all happens around 50-100 times per second in my game, and there’s no noticeable lag. So that’s a good thing.

wait a second. he’s passing the entire game world into most of his functions, and that’s still cool?

what the hell?

i’ve spent most of my past four years at stanford — and, hell, most of my life as a programmer — learning that this is basically the complete opposite of what you’re ever supposed to do, ever, because on the face of it, it looks like you’re wasting more resources than a Goddamned Hummer. i thought it was supposed to be caches all the way down! aren’t you supposed to cache the game world and just make incremental updates to the parts that change? doesn’t throwing away a hundred copies a second waste incredible amounts of memory and put holes in the ozone layer?

but, soft!

Our programming model does not have to match how our computer works.

oh! hm. right. the mother of all abstractions.

i haven’t done any functional programming yet, but thanks to brian and scott, i’m making it a resolution to finally learn me a haskell before the year’s out. or a clojure, or something.

update:

in the comment thread on proggit, user radarsat1 has a really eloquently put take on the situation:

[one] misconception [here] is that seeing the whole world is bad. Think about why it’s considered bad practice in C or C++ to have singletons and globals, and to pass in large data structures to functions. It’s because each and every function has the potential to modify the world, and therefore strange interactions can occur that later on will be difficult to trace and think about. In purely functional programming, if a function is passed the whole world, big deal.. it’s read-only. What harm can it do? We can know apriori that it’s not going to upset any major data structures or cause race conditions, etc. So passing in the world is not as big a problem in FP.

Personally I’m also very young in my understanding of FP, but I’m slowly learning to think functionally, and you really need to reconsider many of the “golden rules” about how to manage your code that you’ve learned to abide in order to keep things tractable in the long term. Many of them simply cease to apply when you have immutability on your side.

you - yes, you! - can teach yourself how to program.

Tue, 02 Mar 2010 12:07:00 -0800

in the beginning was the question

do you think it’s possible to teach yourself programming?

a friend of mine asked me that a couple of nights ago, to my delight. my answer, of course, was

hell yes!

as a matter of fact, any self-respecting programmer you’ve ever met is mostly self-taught. that’s because as soon as a neophyte starts coding, they find out that it’s really fun, so they want to learn how to do more of it and get better at it, and they find out pretty quick that there are plenty of ways to do that on their own, teachers be damned. as another friend of mine put it after spending a week learning how to program for the first time:

yeah, now i get why you guys code all the time. it’s like playing a bunch of little games!

programming’s all about solving problems and making things. hell, it’s about making things that solve problems! how cool is that?

the problem

the only catch was, i didn’t have a handy “learn to program!” link to direct her to. there are a lot of great resources out there, but i’ve never found that one single link that i felt really gave the right amount of information to someone starting completely from scratch.

the “completely from scratch” part is really important. most of the intro-to-programming guides i’ve seen are written for an audience of born techies who don’t mind spending a couple hours learning about the lambda calculus. most of the people who’ve asked me how to teach themselves to program are not in that demographic.

the solution

so, i sent her an email, which i hope she’ll find useful. she’s not the first person to ask me the question that spawned all this, so i figured that i might know some other folks who might find it useful too, which brings us to this post. so, without further ado:

the email

programming is the best.

once there was a man known to the world only as _why the lucky stiff. his main goal in life was to teach as many people as possible how much fun coding is. he was a hero of mine, and i can tell you all about him sometime, but for now all you need to know is that he produced a bunch of projects related to learning about programming.

i’d recommend you start your career with _why’s poignant guide to ruby. he was a pretty insane person, so it starts off a little weird, but just trust me on this, it is worth reading and is one of the best intros to coding i’ve seen. if you have any questions about anything you read that you don’t quite get, ask me! i love to talk about this shit.

if you’d like a slightly more interactive learning-thing, you should check out try ruby , another of _why’s projects. just go there and type “help” like it says in the info-box-thing, and it’ll start a nice little walkthrough which lets you try out some actual ruby code.

you’re gonna be learning how to write ruby code through those resources. ruby is a programming language, and is one of many; it’s great for learning on because it closely resembles english most of the time, but it’s powerful enough that a bunch of really big web sites and other projects are written in it.

if you’d like to try writing some ruby code on your own after making it through the first five or ten minutes of either of those links, you can!

you’re a college student, so i know you’ve got a laptop running mac os x - bring up spotlight (the magnifying glass in the top-right corner of your screen) and type “terminal”. open the terminal program that shows up in the search results. now, just type “irb” without the quotes, and hit enter.

irb means “interactive ruby,” and when you run it, you’re all of a sudden in ruby-land! at this point, you can write ruby code to tell your computer to do anything you want, and it’ll do it! for example: >> 5.times { puts "reba is learning to code!" } reba is learning to code! reba is learning to code! reba is learning to code! reba is learning to code! reba is learning to code! => 5 >> "reba!".upcase => "REBA!" mess around, try out the concepts you learn in those links. try assigning stuff to variables, try writing functions. for example:>> def hello(name) >> puts "hello " + name + "!" >> end => nil >> hello('reba') hello reba! => nilcoding is all about taking small building blocks like these and making really cool things out of them, which you’ll see if you read the poignant guide. matter of fact, i’d be willing to bet that we could take the concepts you learn in the first three or four chapters of the poignant guide and immediately apply them to make little ruby scripts that’ll improve your life by doing chores for you: calculating reimbursement checks, figuring out the perfect classes to take so you maximize the amount of fun you have while minimizing the amount of work you do, that sort of thing.

so: check out those places, mess around in irb, let me know when you run out of things to do or if those links aren’t making sense to you or if there’s anything else you’re curious about!

programming is the best.

the moral

that all applies to you, too, dear reader. check out those links, try playing around with irb, and for god’s sake contact me if you have any trouble. if you’d like to move past the examples in the poignant guide and start writing handy little programs of your very own, i might even write a follow-up to this post with some more advice. stranger things have happened.

"What if the reason you engage in practical activities has nothing to do with your ability to reason,..."

Tue, 02 Mar 2010 11:02:39 -0800

“What if the reason you engage in practical activities has nothing to do with your ability to reason, and everything to do with being lucky that your particular brand of crazy has some utility?”

- http://dilbert.com/blog/entry/crazy_or_disciplined/

http://wvs.topleftpixel.com/10/02/26/

Fri, 26 Feb 2010 17:38:00 -0800

http://wvs.topleftpixel.com/10/02/26/

snippets from the weekly stanford CS job digest

Fri, 26 Feb 2010 10:51:22 -0800

this is probably gonna be a regular thing. i think i’m gonna be grabbing chunks that make me smile from this weekly email i get and posting them here; i’ll refrain from commentary.

Job Requirement # 2010-001

The ideal candidate is a current student in an engineering/computer science field, with strong expertise in C++, Data Structures, Socket and Thread Programming.

Apart from the great experience and a chance to contribute and grow with a start-up at its early stage, we would provide monetary compensation for your efforts (paid on an hourly basis).

You will be compensated with equity, but we are also willing to consider a negotiable fixed fee / cash compensation. The position does not currently offer a salary, so it will be up to us, as a team to create the revenue needed to pay ourselves.

The Applications Developer is responsible for the development, maintenance, support, secure coding practices and enhancement of Autonomy Interwoven business applications under the direction of the Manager of Corporate Applications

Experience required:

* Excellent Verbal and Written Skills – must be able to demonstrate.

How to use rsync to upload big files to HostGator boxes

Thu, 25 Feb 2010 00:18:00 -0800

tossing a few large .zips up on my HostGator account, and Fetch just isn’t cutting it, my connection drops every so often and the transfer has to start all over again. scp won’t do the trick either.

the secret sauce: rsync —partial, with —rsh=’ssh -p2222’ because HostGator uses port 2222 (rather than the standard port 22) for SSH connections. an example:

[jrheard@jrheard:~]$rsync --partial --progress --rsh='ssh -p2222' very_large_archive.zip jrheard@jrheard.com:~/www/music/

de-licious.

Video

Tue, 23 Feb 2010 16:35:39 -0800