Moved (to Posterous). Giving away GoodBaad

It's been a long time since my last post, so to give myself a bit of a boost I've decided to move the blog to Posterous (sorry if this causes a flurry of reposts in you RSS reader). This should make posting a little easier and means I can get rid of my VPS server and save a few pennies.

I've also decided that I need an outlet for the code I produce out of working hours. Therefore I've started a GitHub account so that others can benefit from the work I've done.

The first chunk of code is my GoodBaad web-app that was started around 2 years ago. It was a lot of fun and was a chance to try my hand at building a scalable PHP site from scratch (year's of working with frameworks like CakePHP left me with a feeling that I needed to understand the process of building one from scratch).  Although it's a bit old now I think I learned a great deal about being economical with code and building small flexible units that were short, easy to understand and ran quickly. Some rough benchmarks on my workstation lead me to think  that this stack could out-perform the equivalent CakePHP stack by around 10 times. These gains were done without any data abstraction (I preferred raw SQL)  or a templating engine (PHP is a templating engine if you ask me), and the app didn't need page caching as it was pretty dynamic (and you could always use memcached if it became a requirement) .

The core MVC was loosely based on ideas described in Rasmus Lerdorf's no-framework PHP MVC framework blog post. I agreed that many frameworks simply bring too much code to a application and for me this hid the most interesting parts of writing code. So, the result was an MVC app that sits somewhere between the no-framework stack and something larger like Cake.

So, if you'd like to run your own version of GoodBaad or need a lightweight PHP MVC stack you can take a look at mine for inspiration:

GoodBaad on github

That's all for now, I'll be posting some functional programming stuff shortly (expect plenty of Lisp/Scheme/Clojure and JavaScript!).

"The more people do, the more society develops, the more problems arise."

The title of this post is an excerpt (from an excerpt) of a quote by Masanobu Fukuoka (posted by Tom Hodgkinson on The Idler blog). Masanobu Fukuoka was a soil scientist who turned his back on the perceived wisdom of modern agricultural practices to develop a technique of "natural farming" that more closely mimics nature.

This approach appeals to me. As a developer it is a core skill to innovate and find solutions to problems. Often these are solutions built on top of other solutions. By taking a step back and viewing the developments from a broader perspective the solution may be to cut away "innovations" rather than adding to them.

Running Shoe Technology Goes Full Circle

I have recently become the owner of a pair of Vibram FiveFingers shoes. These are basically a thin covering for the soles of my feet allowing me to walk and run "barefoot" (the actual soles of my feet are far too soft to be practical).

I came across FiveFingers in the comments for this article about how the invention of cushioned shoes has changed the way humans walk and run. Sports shoes have lead to many humans walking and running in a fashion that differs from the way the foot evolved to be used. As a result many runners (tired of injuries associated with landing on the heel of the foot) are using "barefoot" training methods to keep their bodies healthy.

Here's a rough illustration of how barefoot running has come back around:

  • Humans evolved to walk on their hind legs, their feet acting a a shock absorber along with the ankles an knees
  • Shoes similar to huarache sandals arrive allowing the wearer to withstand uncomfortable surfaces
  • Fashion and innovation combine leading the majority of the developed world to wear shoes with some sort of heal
  • Around 1972 modern athletic shoes are invented, coinciding with the growth of sports sciences
  • Today more people than ever suffer from sport and posture related injuries which need corrective orthopedic treatments. Shoes like the MBT are invented to "correct" the foot and allow us to walk "naturally"

It seems that many of the claims that are made for the benefits of orthopedic insoles and MBT shoes are mirrored by simply wearing no shoes at all.

How about software and programming?

One example of the over-engineering that goes on in software development industry might be: templateing engines creeping into PHP.

PHP started life as a way to customise static web pages (e.g. adding today's date or simple hit counters), it then evolved into a way to manage page layouts and behaviour, effectively an interactive templating language. Further evolution has lead to PHP becoming a full blown Object Oriented Language, and with this development have come various frameworks that improve code reuse and working practices.

Then came an innovation that (in my eyes) went too far, templating languages were introduced to make them manageability of page layout and data representation easier. This meant more code to interpret them and therefore further load on the system running it.

One of PHP's core appeals is its ability to mesh directly with HTML, a language truly intertwined with the web; templating seemed an unnecessary step away from that.

For some applications abstraction layers may be a step too far especially when it is an abstraction based on an abstraction.

In a broader sense this could apply to software in the form of feature creep.

I'm not saying that innovation is a bad thing, far from it. But, it may make sense to look at the whole process and in some cases whittle it back down to provide the best solution.

To conclude the rambling

It is a paradox. If you define the success of a species by ability to bend the environment around to their will, then humans most productive trait is the desire to innovate.

However, innovation upon innovation can lead to imperfections and further problems (which in turn will need further solutions).

A valuable skill, to those who's vocation it is to innovate (software developers especially), may be to recognise when a problem can be solved by removing a layer of complication rather than adding yet another "solution".

Which Language Combination?

I'm trying to be a better programmer. I have been writing and learning as much code as I can, the benefits are a threefold positive feedback loop:

  • Learning is, in itself, very pleasurable (it makes you feel clever, which is nice)
  • New knowledge transfers to, and enhances, existing skills
  • New tools and skills provide a greater resource for providing solutions

But, how does a developer know which skills to learn and which languages are best to learn with? (Assuming that you're learning for fun rather than as a work requirement.)

If Google is a good bench mark (assuming that they can take their pick of the best developers), one requirement (taken from an ad for a Software Engineer) asks for:

Fluency in two or more of C, C++, Java, Shell, PHP, Perl or Python.

If you are specifically aiming for a career at Google then making sure you are fluent in two major languages is probably the way to go, and there is certainly no downside to a developer broadening their skill set. In my experience, each foray into new languages has improved my PHP, and in turn, I feel pretty comfortable in Ruby and Python.

When deciding to learn something new: you could argue that PHP, Ruby, Python, Perl etc... are pretty much different brands of the same tool (procedural, interpreted languages). It's great to have the option to use each, but these languages are not necessarily complimentary. It is unlikely that you would write a system with a combination of PHP, Python and Ruby. But, it is conceivable that you could have a project requiring C, PHP and Erlang skills.

Rather than learning interchangable skills, complementary languages might offer the opportunity to become a more rounded developer:

  • Interpreted, procedural (PHP, Ruby, Python, JS)
  • Compiled, procedural (C/C++, Java)
  • Functional, concurrent (Erlang, Scala, F#)

Assuming that side stepping between similar languages is easier than picking up a completely new concept: it would seem learning these varied skills opens up more possibilities.

Sometimes I worry that by learning a particular, none mainstream, language (like Erlang) I could be backing the "wrong horse". However, bearing in mind my earlier ramblings, there's little downside to picking up a new way of doing things, for example Elang may not prove popular, but the functional approach will give me a head start learning F# or Scala.

This slide from Painless Payment Processing by Erik Stenman points out other benefits of learning more "unusual" languages:

Nice paradox:

The lack of Erlang programmers makes it easier for us to find great programmers.

  • There are many great C and Java programmers, I'm sure, but they are hidden by hordes of mediocre programmers.
  • Programmers who know a functional programming language are often passionate about programming.
  • Passionate programmers makes Great Programmers

Or am I talking nonsense?

Goodbye Browsers: What Next?

It has been mooted fairly often: that the web may be breaking free from the browser, and I'm enclined to agree. So what does this mean to developers and how will this effect the web development process (and my first Erlang Web App)?

I'm currently upgrading a site that will need dynamic elements and interactivety intertwined with flat HTML files in a standard directory structure. During the technical design process it is also becomming apparent that these dynamic features will need to be distributed over various media and platforms.

None of these are unusual requirements and I've been developing similar projects for a number of years. However, my approach is beggining to change. Previously I would have built the whole system with my PHP framework of choice (CakePHP) and used it's inbuilt Views to publish the content in various formats (e.g. HTML and RSS), I'm now rethinking.

Our content needs to spread onto the ever widening array of platforms and media, so our publishing methods need to become more flexible. The solution I'm favouring is to publish content over some sort of web service or API and have the client platform, what ever it may be, interpret (and possibly redistribute) it. The RDF Standard is one format that is gaining popularity, especially as semantic technologies become more mainstream.

These principals are probably further advanced in the realm of Web Applications: it is becoming common place to produce an API allowing other platforms to interact with them. Turning the app into a platform (via an API) allows other's to build on top in turn engraining it into their business model (For example Good Baad relies on 5 or 6 different APIs). Google have been providing useful APIs for years and the results are plain to see.

It seem likely to me that the development process may change as a result. Currently an web app or content is built first as a website and then expanded into and API or feed. It seems more practical to build the API first and then this can be adapted to other platforms.

For example: I moved my blog consumption to Google Reader, now I rarely see the sites that the content originates from; I interact with blogs entirely via their RSS feed over the platform of my choosing. With web applications it seems like the same is becoming true, very few people interact with Twitter directly, but prefer to interact via the API (3rd party apps is one key to their phenomenal growth). And, if concepts like Ubiquity take hold: the web may break free of the browser entirely.

And this brings me around to my Erlang project: I'm still mulling over ideas, but it seems to me that a real-time app that is purely an API would be a very good first project. Also, Erlang seems very well suited to the Web API building task, e.g: from a data distribution view point Couch DB is pretty much all you need.

Over the next few weeks I'll build a simple Erlang app that interacts with the web over an API, it's possible I won't need to worry about building a client at all.

Erlang FizzBuzz Showdown (pt2)

In my last post I documented some of the steps involved in building a simple FizzBuzz script in Erlang and compared the code to its equivalents in Python, Ruby and PHP. So far we've simply covered using a tail-recursive function to creat a list of numbers 1-100.

The next problem to tackle is checking whether a number is: just a number, a Fizz (divisible by 3), a Buzz (divisible by 5) or a FizzBuzz (divisible by 3 and 5). As I'm in the habit of using if statements to perform checks (I'm primarily a PHP developer) my first attempt looked like this:

fizzbuzz(I) ->
    if
        ((I rem 3 =:= 0 ) and (I rem 5 =:= 0 )) ->
            fizzbuzz;
        I rem 3 =:= 0 ->
            fizz;
        I rem 5 =:= 0 ->
            buzz;
        true ->
            I
    end.

This works just fine, but isn't a good example of writing Erlang in a functional manner, this is more like imperative programming in style. For example, it is not dissimilar to this python function:

# python

def fizzbuzz(i):
  if i % 3 == 0 and i % 5 == 0:
    return 'Fizzbuzz'
        
  elif i % 3 == 0:
    return 'Fizz'
        
  elif i % 5 == 0:
    return 'Buzz'
        
  else:
    return i

In Erlang (and I assume functional programming in general) Guards rather than if statements are far more appropriate. So, using the multiple entry points for a function and combining them with a guard I ended up with this:

fizzbuzz(N) when N rem 3 == 0, N rem 5 == 0 -> "FizzBuzz";
fizzbuzz(N) when N rem 3 == 0 -> "Fizz";
fizzbuzz(N) when N rem 5 == 0 -> "Buzz";
fizzbuzz(N) -> N.

Once I'd taken on board the functional approach the FizzBuzz function has not lost any readability (if anything this is easier to follow) and has become more compact and efficient. From what I've seen so far: Erlang programs rarely need if statements.

The next stage in the FizzBuzz program is to combine this with my earlier range function and loop through the numbers 1-100. Attempt 1 worked like so:

combine(To) ->
  F = (fun(I) ->  fizzbuzz(I) end),
  lists:map(F,range(1,To,[])).

Again, this works: the function combine takes in the number to be counted up to (e.g. 100), then a Fun is created (basically a mini function that can be passed into other functions as an argument) that is a copy of fizzbuzz, and then uses the inbuilt module/function lists/map to apply it to each number in the list returning a new list with the appropriate changes. But again this can be condensed to make better use of Elang's functional syntax.

combine(To) ->
  [fizzbuzz(X) || X 

The end result is the same but the function now uses a pattern matching/balancing style syntax to produce the new list. The process breaks down like this:
  • [] is the list notation, and using the pipes to separate the two haves of the process ([X || X ) means that the result of the right hand site is passed into the left (as X, formed by breaking the list down X ), forming a new list one element at a time.
  • The input list is formed using the range function on the right.
  • Then the fizzbuzz function processes the incoming elements of the new list on the left.

And that almost concludes my Erlang FizzBuzz program. However, it is returning a list rather than printing one line at a time. This is the module so far:

-module(fizzbuzz).
-export([fizzbuzz/1,range/3,combine/1]).

fizzbuzz(N) when N rem 3 == 0, N rem 5 == 0 -> "FizzBuzz";
fizzbuzz(N) when N rem 3 == 0 -> "Fizz";
fizzbuzz(N) when N rem 5 == 0 -> "Buzz";
fizzbuzz(N) -> N.

range(To,To,List) -> [To|List];
range(From,To,List) when From  range(From, To -1, [To|List]);
range(From,To,List) when From > To -> List.

combine(To) ->
  [fizzbuzz(X) || X 

And being used in the console:


Eshell V5.6.5  (abort with ^G)
1> c(fizzbuzz).
{ok,fizzbuzz}
2> fizzbuzz:combine(100).
[1,2,"Fizz",4,"Buzz","Fizz",7,8,"Fizz","Buzz",11,"Fizz",13,
 14,"FizzBuzz",16,17,"Fizz",19,"Buzz","Fizz",22,23,"Fizz",
 "Buzz",26,"Fizz",28,29|...]


Finally, this can be condensed one last time by turning the range function into a tool for applying the fizzbuzz function to each number recursively, printing the results to the console as it goes:


-module(fb_final).
-export([fizzbuzz/1,fbr/2]).

fizzbuzz(N) when N rem 3 == 0, N rem 5 == 0 ->
  io:format("FizzBuzz\n");

fizzbuzz(N) when N rem 3 == 0 ->
  io:format("Fizz\n");

fizzbuzz(N) when N rem 5 == 0 ->
  io:format("Buzz\n");

fizzbuzz(N) ->
  io:format([integer_to_list(N) | "\n"]).

fbr(To,To) -> fizzbuzz(To);
fbr(From,To) when From  fizzbuzz(From), fbr(From + 1, To).



And with that I'm pretty pleased. The fb_final module makes good use of the functional approach, is concise but still fairly easy to read. In conclusion I'm enjoying Erlang a great deal.

P.S:
There is one last small modification that makes fbr/range reusable allowing higher order functions to be passed into it:


range(To,To,F) -> F(To);
range(From,To,F) when From  F(From), range(From + 1, To,F).


This executes using the following console script (inserting fizzbuzz into range as a higher order function):


Eshell V5.6.5  (abort with ^G)
1> c(fb_final).
{ok,fb_final}
2> fb_final:range(1,15,fun(N) -> fb_final:fizzbuzz(N) end).
1
2
Fizz
4
Buzz
Fizz
7
8
Fizz
Buzz
11
Fizz
13
14
FizzBuzz
ok



Done and dusted! And here are some other ways of getting FizzBuzz done in a functional style.

Erlang FizzBuzz Showdown (Pt 1)

As I've become a little more familiar with Erlang I decided to leave the set text and test myself with a basic (often seen) programming challenge (FizzBuzz) and compare the solution with the languages I'm more familiar with.

FizzBuzz is a pretty simple set of rules (also known as Bizz Buzz) and is used for children's maths, drinking games and as a rather basic test given to programmers during job interviews (although it is so well known that it probably lost its value for this purpose a while back).

The rules (for programmers) are simple:

  • Print a list of numbers 1 - 100
  • If the number is exactly divisible by 3 print "Fizz"
  • If the number is exactly divisible by 5 print "Buzz"
  • If the number is exactly divisible by both 3 and 5 print "FizzBuzz"
  • Otherwise just print the number

Solving the problem breaks down to:

  • Create a range of numbers 1-100
  • Loop through each number and check it against the conditions
  • Print out the result
  • Repeat until 100

So far so simple, it is a pretty straight forward task that can be solved in a broad variety of ways.

Here's how I could achieve this in Python:

for i in range(1,101):
  
  if i % 3 == 0 and i % 5 == 0:
    print 'Fizzbuzz'
        
  elif i % 3 == 0:
    print 'Fizz'
        
  elif i % 5 == 0:
    print 'Buzz'
        
  else:
    print i

And PHP:

foreach (range(1,100) as $i)
{
    if (($i % 5 == 0) && ($i % 3 == 0))
    {
        echo "FizzBuzz\n";
    }
    elseif ($i % 3 == 0)
    {
        echo "Fizz\n";
    }
    elseif ($i % 5 == 0)
    {
        echo "Buzz\n";
    }
    else
    {
        echo "$i\n";
    }
}

Of course there are many variations depending on how short and/or "readable" you'd like it to be.

So, when it came to tackling this problem with Erlang I was a little surprised that I couldn't just launch into it. Approaching this problem functionally is a different kettle of fish. While there are built in functions that can help I chose to ignore them in favour of gaining a little more understanding.

In other languages creating a range of numbers was pretty straight forward and barley registered as a task, in Erlang I needed to think a little harder. There is no for or while loop easily to hand, the solution was a tail-recursive function. It might seem bloated to add this function, but if you compare the same function in Ruby, the Erlang function appears better suited to the task (also using tail recursion in none functional languages causes performance/stack issues so this is more for illustration):

% ERLANG

range(To,To,List) -> List ++ [To];
range(From,To,List) when From  range(From + 1, To, List ++ [From]).
#RUBY

def jRange(from,to,list)
  if from == to
    return list 

Erlang functions can have multiple entry points, so in this case the range function has two entry points:
  • One that accepts two identical values, a list and returns a list with the value added.
  • Another that accepts two different values, a list and adds the first value to the list, then calls the same function (itself) adding one to the first value.

As it matches the incoming conditions there is no need for if or else.

The problem with this loop is that it will go on infinitely if From is greater than To, to cover this we need a guard. This is a simple check that From is less than To:

range(To,To,List) -> [To|List];
range(From,To,List) when From  range(From, To -1, [To|List]).

And now we have an operational "range" function similar to those seen in Python, Ruby and PHP - however, Elang FizzBuzz can use it in various different ways. Another small change was to alter the way the list builds up, using List ++ [From] isn't very efficient and [To|List] does the job more elegantly.

In the next post I'll conclude building a FizzBuzz program (mistakes, solutions and better-practice included), check back tomorrow or subscribe to keep up to date.

Part 2 >>