Type Aliases for Functions - Wat?

I'm still buried deep in Functional Programming Scala.  It's an excellent book, but it does have a habit of occasionally introducing new topics a little out of the blue.  This morning: Type Aliases.

It means that when we have something like this:

And we then make a type alias like this:

We can do this:

Woah, what just happened there?  Lets rewind. The first clue we need is in the name. Type aliases allow us to give a new name (or an alias, as in "Braddock alias Thorne" - someone who is known by more than one name) to an existing type.  Why? For readability and to save on the typing (the fingers-to-keyboard kind).

E.g. by defining:

You can now write DataTypeMap anywhere in your code that you'd normally write Map[String. DataTypeAsJson] (thanks to Alvin Alexander for his notes which I stole this from).

That's nice and clear, but it doesn't seem to get us all the way to type Rand[+A] = RNG => (A, RNG).  Why not?  The missing piece of the puzzle is down to the fact that here we're aliasing a function type. It's saying that "the type Rand[+A] is an alias for an action which depends on an RNG type, and from that it generates a new A type and returns it in a tuple along with a new RNG transitioned to the new state." (I paraphrase a little from the description in the book. And to be fair to the authors, they say its a function type later on in the same page.  I'm just very slow on the uptake.)

Lets take this knowledge and scroll back.  The nextInt function is untouched in all of this - we're simply dancing around it with the second, type alias decoration.  The third bit is the most interesting.

So what's happening there?  Stepping through it, we can see we're making a new value called (slightly confusingly I'd argue) "int" of our new type Rand[Int].  Now remember, in Scala functions are objects too, and here we're seeing it in action - we're pointing our new val int at the nextInt function.  With this in mind it makes a lot of sense that the type of int is a shorthand for RNG => (A, RNG) because it refers to a function (nextInt) to which you provide an RNG, and get back a tuple of an Int and a new RNG with the state transitioned.

When you look at it like that, it's actually quite beautiful.

Asides

• The first time I came across type aliases I was even less prepared - there is one in starter code for the first Functional Programming in Scala exercise.
• Interested in the Haskell equivalent? It seems to me (please ahout if I'm wrong) that Haskell's Type Synonyms are a pretty close parallel.
• Want to read more? Twitter's Effective Scala seems like a good place to start

Tracing, Step by Step

In this post I want to introduce the simple yet powerful technique of tracing.  It's one I'm using more and more to help me get my head around the subtleties of recursion and in the process get a deeper understanding of the factors at play in this type of algorithm.

We'll take as our first example a piece of code from my previous post - a call to a function called foldLeft:

To begin, we next need the internals of this foldLeft function too:

With these two pieces of code in hand, we can begin to trace.  To do so we ask ourselves, 'what does our starting piece of code become if we replace the call the function, with the elements of the function which will be executed in the call?'  So, given this for the first line of our trace:

We end up with a second trace line as follows:

Let's work through this slowly - what happened?  It's actually a combination of micro steps. When we make the call, 'as' is List(1,2,3), 'b' is 0, and 'f' is (_ + _). Consequently, the pattern match will trigger on 'case h :: t', and we're left with a new call to foldLeft.

This new foldLeft call will have parameters based on the application of the right hand side of the case statement:

't' is the tail of the original 'as' List (now expressed in terms of '::' (or "Cons") - 2 :: 3 :: Nil,
'b' is now the application of the function f to the original value of 'b' and the value of the head of 'as' - f(0, 1) or (0 + 1),
and 'f' is still 'f' - (_ + _)

ASIDE: For some (e.g. me as I was learning this technique) this is too big a jump the first time.  In this case you can trace out intermediate steps, which will give you:

Before we continue, applying the technique again on our new foldLeft input there is an important thing to note (and an illustration of a powerful benefit of the tracing technique).  Note on the last line that we've *not* collpased the function 'f' - we have '(0 + 1)' rather than '1'.  Why not? We have all the information to hand and we could have done the maths and nothing would have broken. The reason we leave it is a convention of tracing - we are aiming simply to reduce all the function calls to their simple equivalents. It is nice to leave the simple elements of a trace explicit until all the calls elsewhere have been made so that you can see the resulting function in the raw, and complete - the one that will be executed when the code runs.

Lets move on.  When we left off, we had another foldLeft call to trace:

So lets apply the same logic as last time and see what we get.  Again we trigger on the second case statement, giving the following values to the right-hand-side:

't' is the tail of the 'as' in the current context - 3 :: Nil,
'b' is the application of the function f to the current value of b and the head of 'as' - f((0 + 1), 2), or ((0 + 1) + 2),
and again 'f' is still 'f' - (_ + _)

Mapping this onto the right-hand-side call to foldLeft gives the next line of trace as follows:

We need to keep going - we still have a foldLeft call to expand.  We'll not labour the point of this one (its the same as the previous step). Now we have (I've omitted the micro-steps for clarity and brevity):

This leaves with yet another call to foldLeft, but this time we're going to trigger the first case statement. What happens in this circumstance? We forget about everything else, and simply return b - which in our case is '(((0 + 1) + 2) + 3)'. Let's add it to the bottom as usual:

And with that we've fallen out the bottom of foldLeft - no more calls, just simple maths.  Some people stop here - our result is now clear. Others like to add the final line of the trace as the reslt of this equation. Its kind of up to you. I'm not going to bother to save you scrolling even more, and I don't want to patronise you.

Given all this, below are some more traces which you can look at to embed this technique fully.  I encourage you to do some of your own. It really helps.

First up is a 'product' function which simply calls foldLeft with a multiplicative function.

Given: def product(l: List[Double]) = foldLeft(l, 1.0) (_ * _)
When:  product(List(10.0,23.0,96.0))
Then:

Lets move from foldLefts to a foldRight.  This one is from Functional Programming in Scala:

Given: def foldRight[A,B](as: List[A], b: B)(f: (A, B) => B): B =
        as match {
            case Nil      => b
            case x :: xs  => f(x, foldRight(xs, b)(f))
        }
When: foldRight(List(1,2,3), Nil) (List(_,_)) 
Then:

Finally here's another foldRight.  Note that this one is tricksy, due to the slightly unusual higher order function that's used. It's instructive here to see the trace in action. (Note: I've added the function calls but they don't compile. You could fix this by substituting the anonymous function in the example for a named one.)

Given: def foldRight[A,B](as: List[A], b: B)(f: (A, B) => B): B =
        as match {
            case Nil      => b
            case x :: xs  => f(x, foldRight(xs, b)(f))
        }
And: def length[A](as: List[A]): Int = foldRight(as, 0)((_, acc) => acc + 1)
When:  length(List (5,6,7))
Then:

Power Tips

Before we close its worth mentioning a few tips to help you get the most from this learning tool:

• formatting is your friend - as you can see from the above examples, use spaces liberally to make clear to yourself how things are progressing
• so is your IDE - every trace line should parse and compile, and give the same result (though you might have to replace anonymous functions with named ones.)
• use this when thinking about execution-time, and tail recursion, and design of your own algorithms - sometimes the functional style can be hard to make the jump to when you're coming from an imperative background.  This technique helps significantly. By mapping it out line by line you can check and understand your algorithm, see any potentials problems it has, and see why or why not its tail recursive.

The Slow Boat to Folding

I'm back at the Scala-face, working my way through Functional Programming in Scala by Paul Chiusano and Rúnar Bjarnason.

I'm still taking things slowly.  The current task was grokking folding.  Why? As Miran Lipovača says in Learn You A Haskell:

"Folds can be used to implement any function where you traverse a list once, element by element, and then return something based on that. Whenever you want to traverse a list to return something, chances are you want a fold. That's why folds are, along with maps and filters, one of the most useful types of functions in functional programming." 

(I've substituted italics for the authors own bold text)

My journey to this post has been a circuitous one; false assumptions, based on semantics from my primary idiom (imperative, non-maths-based programming) having led me astray.  I know that there's a lot out there already about folding in Scala and so I've attempted to avoid repeating that. Instead I've mapped out the steps which eventually helped me wrap my head around things in a way that stuck.

Why I got confused (part 1) - worrying about "starting at the left or the right and folding it in"

Before we dive into the "how I grokked it", I think it pays to define why I got confused to begin with. My first mistake was to think about these pair of functions in terms of how they ate collections. I had some bonkers origami-style image in mind: a long-piece of paper, with the function starting at one end and folding it up concertina style, element by element.

I know now I shouldn't have bothered. I'd advise you yo do the same because it'll quite possibly be confusing you too. (For starters you're making a new, separate and individual thing from the fold. You're not simply copying the collection, applying a function to each element in turn - that's mapping.)

Why I got confused (part 2) - worrying about tail recursion

And while we're in the habit of dropping things, I also eventually found it useful to stop worrying about the design of the algorithm itself; specifically, "was it tail recursive or not?" Why? It's a level of complexity you don't need in our head right now. It certainly clouded my thinking for far too long. So let it go.

Step 1 - Start thinking about associativity

Now we've freed up all this mental space, what can we put up there in our lovely spacious minds so we can make progress?  The first thing is a nice simple picture of associativity.

Now, if you're like me, you'll already "know" about this topic - it's one of the standard bits you skim when learning a new programming language right? And you skim it because it's easy and obvious, right? So why am I going to bore you with it now? Because it's going to give us our "right" and "left", that's why.

First up is left-associativity.  Say we have a sequence of numbers and operators like so:

1 - 2 + 3 = ?

If I ask you to "add that up" you'll reply "simple, it's 2".  But what happens if we add some parentheses?:

1 - (2 + 3) = ?

You'll then answer, "still simple, it's now -4".  So what changed?  Why the different answer?  Thats because we've changed the order in which you apply the operators.  We've monkey-patched the associativity.

Step 2 - Tackle left-associativity first

What we're doing when we don't have any parens (and possibly aren't really thinking about anything like associativity at all) is using left associativity - it's the default in conventional mathematical notation, and the one we learn without realising it in primary school.  It applies when two operators have the same precedence (which '+' and '-' do), and tells us that when calculating a sequence of operations of equal precedence we should start at the left hand end of the same-precedence run.

We can make it explicit with some more parens:

((1 - 2) + 3) = 2

Brilliant.  Nice and simple.

Step 3 - Tackle right-associativity second

From this, it's easy to guess what this looks like with some explicit, parens-driven right-associativity:

(1 - (2 + 3)) = -4

Nice and simple yet again.

But look, see how I've bolded the parens?  That's the important bit for us as we take the next step.  Go on, look at it a little longer - burn it into your retina a wee bit.  It helps; trust me.

(Note: This has been a ludicrously simplistic overview - if you want to know more, the wikipedia article is nice and parseable, even for non-maths-heads.  Oh, and if I was to write the calculation with proper "right associativity" I'd not need the parens - I'd just write it and say "this is right associative". I put them there however to make us do left-associativity in a "right" way.)

Step 4(a) - Reduce foldLeft

"But wait" I hear you say, "I'm always only adding (or only subtracting) when I foldLeft / foldRight. What does associativity have to do with things?"

Let's put this aside for a minute and come back to folding.  Taking a hint we just got from associativity, let's tackle foldLeft first.  In the answer to Functional Programming in Scala's question 3.10 it looks something like this:

We'll now use this function to fold a simple List, tracing the recursive calls one call at a time:

What do you see? Look at the parentheses. Do they remind you of anything?  Yup, thats it - they look mightily similar to the ones in our previous example in step 2.  What we have some here is some explicit left-associativity.

(Note: I've just assumed here that you know how to trace. Its a great technique to wrap your head round things in the Functional world.  I'm planning on writing a post about it when I get time.  For now, just read it line by line - each is equivalent, but just expands the function calls into the bodies of the function that get executed.)

Step 4(b) - Reduce foldRight

Let's crash on; ther's no point hanging around now we have this nugget.

Here's the simple foldRight from Functional Programming in Scala (listing 3.2) and the associated trace:

The parens should really be jumping out at you now.  Notice anything this time?  Yup, its the same as the previous example in step 3 - this is exlicit (i.e. parens-driven) right-associativity.

Step 5 - See the fact you are starting at the left or right end in the reduced output

Now we're building up a nice bit of "that-makes-sense" momentum lets take the next step.

Remember is step 1 I advised you forget about the "starting from the left (or right) and folding in" bit? We're now in a position to bring this back into the frame.  What might have confused you at the beginning (it confused me anyway) was the fact that the key recursion calls both  (foldLeft and foldRight) start at the head of the List in question and eat their way down to the tail (i.e. both recursively eat away at the collection, one "head" at a time).

However, looking back, we now see we can't start collapsing the fold (i.e. start applying the higher-order function, turning it into its result) until we have the innermost pair of values. In the foldLeft example this is at the left hand side of the traced output, and in foldRight this is at the right hand side.  (If you want some corroboration, see where the seed values are placed in step 4 above (the '0' - on the left in the foldLeft, and on the right in the foldRight.)

With all this in place, you now have the means to see how this fold calculation is built up, and how, when it reaches it's limit (the end of the collection) it can be collapsed again down to a result.

Nice.

Step 6 - See the fact that what is important is why one is trail recursive and one isn't

Before we close, lets continue to ride the momentum of clarity and see what else we can learn.  You may have heard tell of the fact that;

"a call is said to be in tail position if the caller does nothing other than return the value of the recursive call" 

Functional Programming in Scala, Chapter 2, pp.20

What's different is the way the calls are made: foldLeft does it straight, while foldRight does it as the second parameter to a call to the higher order function.  Look; see:

If I tell you that foldLeft is tail-recursive, and foldRight isn't, that should come as no surprise.

In foldRight, we have to apply the function to the result of the recursive call, whereas in foldLeft we have the result there and then and simply pass it back. This is clearly also the source of the "right" and "left"-ness.  The differences in these two expressions is what leads to our parens, and our starting at the right hand end or the left.  It suddenly all comes together.

Taken together it now seems self-evident (to me at least, and hopefully to you if I explained this all clearly enough) how a foldLeft could compile down into a stack-safe loop, while a foldRight would have to use a stack-imperiling non-tail recursive route.

Bonus Extra Step 9 - The order of the params to fold left and fold right is just convention

To close, as we're riding the crest of the grok-wave, it's not a bad idea to see what else we can find to slot into our noggins.

It's not really related to this post apart from the fact that it's to do with fold functions (and we're now in a position to full appreciate it) but how about the order of the params in Haskell's foldLeft and foldRight?

You'll notice if you look it up that the position of the accumulator param differs. It'd be insulting to point out that the order of params in a function doesn't generally matter, so why is this the case? Let's let Miran Lipovača explain again:

"the left fold's binary function has the accumulator as the first parameter and the current value as the second one, the right fold's binary function has the current value as the first parameter and the accumulator as the second one. It kind of makes sense that the right fold has the accumulator on the right, because it folds from the right side"

from Learn You A Haskell

Lovely.