Mapping Forward, Looking Back

I’m taking things slowly.  There’s frequently a lot of meaning in very few words when learning a new (programming) language.  I’m still in Chapter 1 of Scala in Action, Nilanjan Raychaudhuri, considering the most fundamental parts.  What do I know so far?

In Scala:

“Every value is an object, and every operation is a message send.”

Scala in Action, Nilanjan Raychaudhuri, Chapter 1

Therefore what I can write in my code as 1+2 is actually interpreted by Scala as 1.+(2)

I.e. I have a value (1) and it has available functions, one of which is +.  I access these via the dot-notation (.) same as in Java and many other languages.  I then send across the argument-value by placing it within the brackets.

I can also see from this that the rules for identifiers are far more relaxed. In fact, in Scala an identifier can be made up of a sequence of 1-N letters and digits, starting with a letter, or a sequence of operator characters, so we can define methods called <=, or *.

Some more:

“[Even] functions are first class values [their italics]. That means that […] every function is a value, and like any values, you can pass them as parameters and return them from other functions.”

Scala in Action, Nilanjan Raychaudhuri, Chapter 1

NOTE 1: “Every function is a value” and not simply an object.  This is more specific.  That’s an example of what I mean about there frequently being a lot of information in a few words.

NOTE 2: In effect, function values can be sent as messages to other functions which are capable of receiving them.

So to bring it back full circle, I started out on this post from the statement:

“You can assign a function (x: Int) => x + 1 to a val inc and use that (the val) to invoke that function: inc(1)”

Scala in Action, Nilanjan Raychaudhuri, Chapter 1

This had seemed like a leap to me. However, I’m mature enough now to realise that most things which look initially like big leaps, are more likely an indication that I’ve missed something earlier.  That’s what I’ve gone back and put in this post so far.  And so now, reminded of what I should have remembered the first time, does it ring true?

Well, yes, it does.  Looking at the first example, seen as the Scala compiler sees things, what we have here is a direct equivalence, only one function is called + and the other inc.  Perfectly simple.  And all making sense again.

Proceed!

NO Side Effects

I’ve heard of side-effects before. Uncle Bob? He ‘ates ‘em.  I’m totally paid up on the benefits of removing them too. Clean Code. But in the Scala world, they’re more than a nice to have. The core of the language expects you to avoid them, and so, it seems incredibly relevant to state clearly what a side effects are:

“A function or expression is said to have a side effect if, in addition to producing a value [a return type], it modifies some state or has an observable interaction with calling functions or the outside world. A function might modify a global or a static variable, modify one of it’s arguments [without taking a copy first – think about the pitfalls of pass-by-value with objects in Java], raise an exception, write data to a display or file, read data, or call other functions having side effects.”

Scala in Action, Nilanjan Raychaudhuri, Chapter 1

Now that all sounds great.  Especially the implication that, with no side effects you can always determine a program’s behaviour.  But a closer look sends my head reeling. How do I handle exceptional circumstances if I can’t thrown an exception? How do I do any meaningful work if I can’t read from or write to files (or, I guess, other things like network resources)?  I’m guessing there will be some kind of Functional “Get Out of Jail Free” way around this, or into it.  Of everything I’ve encountered so far, this is the biggest mind shift – and one which, when grasped, will definitely have a positive effect on my attempts at Bob-friendly code.

f(x) = y

I think I’m getting to fully understand “map”. (Source of much prior consternation.)  In the carefully crafted introduction to what it is to be a Functional Program, Nilanjan Raychaudhuri (Scala in Action, Chapter 1) introduces the following:

“x has a relationship f with y”

which can also be stated as:

“x maps to y via f”

All of a sudden, in two simple statements, “map” has been used as a verb, in a way which seems memorable to me, and more importantly, in the same way that it’s used in the Scala “map” function.  That counts as a first piton in my book. A good solid one.

Changing Cliff-Faces Mid-Stream

I’ve got a new book – Scala in Action by Nilanjan Raychaudhuri. It means I’m going to calm down in the Koans for now and focus on this more wordy introduction.

I’ve only read the Foreword and a few pages of the first chapter but the first paragraph of Chad Fowler’s contribution already resonated with me:

“You’re standing in front of a huge, steep wall of rock. Your neck is straining as you bend our head back as far as you can to take it all in. If you squint, you can barely see something moving around at the top. There’s probably some really good stuff up there. You heard from people you trust that it’s worth climbing this wall. But, you’re damned sure going to hurt yourself on the ay up. You can already see some of the jagged edges jutting out. And what if it turns out that you don’t like what you see when you get there?”

Scala in Action by Nilanjan Raychaudhuri, Foreword

This didn’t just strike a chord with me, it resonated. I’d already blogged about getting a toe-hold in the language, and the image of a cliff face had been one which had occurred to me on multiple occasions. Chad had also effectively articulated my fears. If I was to do this properly, to learn to program Functionally then I need to do it wholeheartedly. That means, as he points out, I risk not liking what I see when I get to the top. Even if I do, it’s going to go against the grain of much of what I already know and trust which means it’ll hurt. It will most likely hurt a lot.

So why? Why go on with something which Chad says in the very next sentence is “difficult”? Because it’s there, and if it proves that I’m just a Java/OO type of guy, then at least I’ll know better why I am.

Conventions, Conventions

Interesting comment in the last koan of AboutMutableMaps (koan("Mutable maps can be cleared")):

Makes sense.  And it’d be a nice signpost to the reader.  It got me thinking, Scala is a language still gaining in adoption, and changing regularly too; so what are the current style conventions?

Well, I had a look through some old Evernote notes I’d made at the last Java Posse Roundup (attended by quite a few Scala mavens) and I had this bookmark to a document called The Scala Style Guide.  Apparently, its contents are “followed with more or less fervency”.  I think I should give this a read.

Notably, I’d linked there from “Effective Scala” (written by Marius Eriksen out of Twitter) from a session I’d attended about Scala Adoption.  It seems like that might have a treasure trove of information I can plumb later on.

Map.values() returns an Iterable

Hidden in koan("Map values can be iterated") is a comment which piqued my interest:

I cracked out the SBT console to have a better look:

It seems the comment is a little incorrect (myValues is an Iterable rather than an Iterator) but I get the idea.

val me = ? :: !

KOAN SPOILER ALERT
More syntax!  Now we have ‘::’  I’m on the train again with patchy network so I guessed iteratively to figure out this one.  The solution is:

I’ve just remembered, I have a PDF of a roughcut of Programming Scala on my laptop.  A quick “find” has come up with this:

“you can prepend an element to a List using the :: method”

Chapter 3, Section “Precedence Rules" (in my version)

That makes perfect sense.  Again the syntax is not entirely intuitive, but while there is a bit more explanation as to the whys and wherefores of this symbology, I think I’ll stick with this for now.  Actually, one more tiny quote, just to help my “behind the curtain” understanding a little:

“'a' :: list
The […] expression is equivalent to list.::(a)”

Chapter 3, Section “Precedence Rules" (in my version)

UPDATE:  I updated the formatting a little on this after initial publishing. That's all.

Map’s Friends: Reduce and FoldLeft (and a First Toe-Hold)

WARNING: KOAN SPOILER ALERT

This isn’t the original version of this post. There was a first draft which banged on about the impenetrable math-lingo which Scala is steeped in. But that was before I researched and wrote Higher Order Functions <gulp/>. That helped a lot. It felt as if it gave me a toe-hold on the Functional cliff-face. The one hiding just behind the basic syntax. The one which still looms above me.

Now I’m of a much more positive frame of mind. Lets start with reduce.

Now I’m beginning to get over my fear of ‘_’, this was something I could understand.  I wasn’t sure from the Koan code alone initially what this function did, but after a bit of guess-work I got it to work. Again I think I need to come up with a memorable image to associate with this to make it really persist "up there”, but in the meantime I can appreciate how powerful this could be.  I’ll post on my background reading on reduce and foldLeft (and map) later.

Moving on to foldLeft:

This time I could guess pretty easily.  Nice.  Still no mental image springs to mind, but I’m patient.

But what’s this about “currying”?  Patience.  We’ll get there.

Koan(“Lists have many useful methods (but many have funny names)”)

For example, filterNot:

That one needed a comment to tell me what it was doing.

And another example, map (my nemesis):

Now in all circumstances I like the general style – nice and simple. I liked it in Ruby too (or the equivalent), but to me, while “filter” makes sense on its own as something which will give me things which match the filter, filterNot only makes sense like this in light of knowledge of filter.  And I’ve already posted about map before…

Even as I’m writing this I can appreciate that I’m grousing. The above comments even now seem silly and petulant.  Perhaps I’m slowly coming round to the Scala way of thinking.  It is a different mindset. In light of it’s hyper-terse API names, even Ruby is beginning to look verbose.  I didn’t think I’d be saying that two weeks ago.  Its beginning to feel like I need to build up a massive hashmap in my head of common function names and their associated activity. Then when I read some Scala code they’ll light up in my mind with a little associated picture of what they’re doing.  That’s not unlike how Heisig gets into your head too.  It’s surprisingly effective.

So in light of that:

• filter – a sieve, with the resulting things left in it, everything which matched the provided expression-argument
• filterNot – the same sieve, but now we’re looking at the stuff that didn’t get caught in it, the stuff in the sink, the stuff that didn’t match the provided expression-argument
• map – … I’m sorry, I still can’t think of a good mental image that’s going to fit this one.  Not yet. Not without using “apply” heavily and I know that’s reserved for a whole other area of significance…

Higher Order Functions <gulp/>

The time has come to tackle my sparse knowledge of the Functional Idiom head on.  I was googling “computer science map function” to try and get my head round the impending mental traffic accident between me and the Scala Collections APIs.  I found what I was looking for right away but it seemed that to take the first step up to grokking this Functional thing, I should bite the bullet and try and read the Higher-Order Function page.  Whoosh! Right in at the deep end. Let’s go.

“a higher-order function (also functional form, functional or functor) is a function that does at least one of the following:

• take one or more functions as an input
• output a function”

That makes sense.  The Scala map function clearly takes a function as an input parameter (for example).

I’ll skip the next bit as it was gobbldigook (perhaps I’ll be able to make sense of it later down the line) but I picked up the thread again two paragraphs later:

“The map function, found in many functional programming languages, is one example of a higher-order function. It takes as arguments a function f and a list of elements, and as the result, returns a new list with f applied to each element from the list.”

Brilliant.  That makes perfect sense too, and we’re using my personal bugbear as the example.  I’m beginning to see why “map” isn’t such a stumbling block to everyone else. If only I’d studied Computing Science at University… Lets read on.

Right, next came a lot of examples in programming languages I’ve not used (or actively avoid – sorry JavaScript). I’m tempted to write a Scala-equivalent of one of them, but I think it’d be jumping the gun a little.  Instead, lets read the “Alternatives” section as it seems to start speaking my language:

“In object-oriented programming languages that do not support higher-order functions, objects can be an effective substitute. An object's methods act in essence like functions, and a method may accept objects as parameters and produce objects as return values. Objects often carry added run-time overhead compared to pure functions, however, and added boilerplate code for defining and instantiating an object and its method(s).”

Now that makes perfect sense to me.  It’s one of the reasons I’ve wanted to get on board with Scala.  It feels like we’re getting somewhere.  In the next post I’ll dip back down to “Map” and perhaps even “Fold” and see what these words mean to these crazy computing scientists.

The Incal is to Scala what Why’s Poignant Guide is to Ruby

If Ruby has a comic, they why not Scala? The Incal is a graphic novel collaboration between Alejandro Jodorowski and the French Comic artist Moebius.  Set in the future, on first read, it makes little sense at all. The main protagonist, an everyman called John DiFool is pulled into an intergalactic, trans-dimensional, intra-personal journey of epic proportions.  He frequently has no idea what is going on, and is put through some terrible, but ultimately transformative experiences on his way to ultimate enlightenment.

At the moment, it feels a bit like that with me and Scala, without the ultimate enlightenment; but then in The Incal, that’s hard fought, and utterly transformative. 

Chunky bacon!

Images from: http://wondersinthedark.wordpress.com/2012/03/31/when-the-fall-is-all-thats-left-jodorowsky-and-moebius-the-incal-on-page-and-screen/

_Just _Get _Over _It (aka _Tuples _1_0_1)

Coming from Java, the frequent use of ‘_’ freaks me out.  E.g. when accessing Tuple members

In this case I know (from the Fast Track to Scala training) that under the covers we have a Tuple2 object, with methods called ‘._1’ and ‘._2’.  (You can see this if you use the REPL / SBT console, and try TAB-completion on the newly created “tuple” object.)

I’m resisting the urge to call the equivalent of “get1()” and “get2()” in Java as I know that’s not really embracing the Scala way of doing things.  I’m trying really hard to move towards the light, really I am.

I’m also trying to get my head round the “(“apple”, “dog”) syntax. This seems to be a Tuple-literal, but I’ll need to look into it.  What I do know is that this passes:

And this does too (perhaps less surprisingly):

Bit of a conceptual leap from the Koans there…

 

Specially Selected Bonus Prize

A little bit of additional info for the keen amongst us. Scala By Example he say:

“Tuples are so convenient that Scala defines special syntax for them. To form a tuple with n elements x1, . . . , xn one can write (x1, . . . , xn). This is equivalent to Tuplen(x1, . . . , xn).”  Now why couldn’t they have used that ‘…’ goodness in Ranges too?

Ranges—, Ranges++

RETRACTION ALERT: See the very end for the good news

I might be jumping the gun on this one (I was, see the “Retraction” at the end of this post) but I think I’ve come across my first Scala niggle.  Having range support build into a language is really nice, and fair play to Scala for having it.  I’m not sure I like the syntax however (and this might be where I’m missing something – please comment if I have).  What I mean is this:

and this:

That’s great, but in Ruby (and IIRC, Groovy) the same end (albeit without the immutability) is achieved by:

Now I’ve done some small-scale (i.e. not exhaustive) digging, and I know there is also “range” support built into many of the collections classes and the Iterator class too.  My beef however is around legibility, and “least surprise”, especially with the second Scala example.  To my eyes, when I see the Ruby examples in code, it just reads.  That’s what I want from a modern language.  It’s not all doom and gloom however. One thing I disliked about Ruby ranges (and again I think Groovy ranges too) was the exclusive version of the syntax:

Spot the difference? No? ITS ONE DOT! That’s a potential source of bugs if ever I saw one.  Compare with Scala:

Now you could argue that to have the inclusive set should be the default, but then you should argue that array indexes should start at 1 – the battle is lost. but this, I like. One final thing; I know I have the power available to me in Scala to allow me to write a function which will work like the Ruby syntax (in fact it’d be a good exercise, and one I’ll aim to come back to) but should I have to?  Not if I can avoid it.

Retraction

I just got to the AboutLists Koan. Hidden away in koan("You can create a list from a range") is a sweet little Range literal:

That’s nice. I like that even more than the Ruby syntax. I should have kept my big mouth shut. Well done Scala, well done.

SBT… ~!

SBT: Definitely not “simple”, and not like anything I’ve ever used before, but it’s powerful.  I like that I can ask it to create .idea or .project / .classpath / .* (i.e. Eclipse) files, and I really like the ‘~’ command. E.g. if I start sbt in the Scala Koans project and then type “~ test-only org.functionalkoans.forscala.Koans” then it’ll run the tests, but rather than finishing, it’ll sit there, waiting for a file to change, and when it detects a change it’ll recompile and re-run, automatically.  Spooky to watch when using IDEA which doesn’t even trust me to save my own files and so does it for me.

I wonder how it’ll work with a TDD approach?

Class Parameters

I’m on the AboutClasses koan.  So far Scala is holding up its end of the promise of terseness.  For example:

and the immutable equivalent:

and if you simply want a private field:

So far, so impressed.

The Scala Koans

While I’m waiting for the Kindle version of Atomic Scala to be released, I’m going to work my way slowly through the Scala Koans.  I got up and running really qiuckly, but the instructions at http://www.scalakoans.org/installation are a little out of date.  Here’s what worked for me (on Windows 7, with IntelliJ Idea 12.1.3):

1. Download the Koans
2. Unzip them somewhere
3. Edit koans.bat so it looks like this:
4. Run koans.bat (if you’re behind a firewall, you’ll need to set your proxy) This will download SBT, the Scala Simple Build Tool and start it. You’ll be left at the SBT prompt: “>”
5. Compile the tests: “> test:compile”
6. Start the tests running constantly: “> ~test-only org.functionalkoans.forscala.Koans” You will see your first task: “Please meditate on Koan …” This is your cue to start fixing the koans, and learning Scala and ScalaTest
7. Start up IntelliJ IDEA
8. Import a project – create it from existing sources. To do this just point IDEA at the unzipped archive. It should guess everything.  Make sure you use JDK 1.6 as 1.7 isn;t currently totally supported
9. Find the scala test you’re being prompted to fix (it’ll most likely be dmarsh-scalakoansexercises-8d7da93dd5f2\src\test\scala\org\functionalkoans\forscala\AboutAsserts.scala)
10. Fix the first broken test and switch back to your CMD window where SBT is running
11. You’ll see it announce “Compiling 1 Scala source to …” and if you’ve fixed the test you’ll get a new prompt to meditate on the next one.
12. Rinse. Repeat.

Fast Track to Scala?

Lets be clear about something from the very beginning - I’m not bright.  I’m very enthusiastic, and that can get you far, but I’m not bright.  Why is this relevant?  We’ll see in a minute…

But let’s also be clear about something else; I reckon I’m a pretty good Java developer. If I was to subcategorise myself, I’d say I was a pretty good Corporate Java developer with good instincts in the Clean Code, Test Driven Design, KISS / YAGNI departments. 

I’d also say that I “get” Object Oriented Programming. By that I mean I can remember the moment when the whole classes and objects concept fell into place in my head and I had a moment of clarity which persists to this day.  It just makes sense.

I’d even go so far as to say that when I don’t grok a piece of Java code pretty quickly, it’s because there is some dodgy design, or a framework has imposed it’s view of the world too strongly, obscuring the clear concepts of an expressive OO design.

But I want to learn a new language.  Why? Well, I’ve done this before. I learned Ruby previously (and blogged my process just as I am doing here) and I less explicitly learned Groovy which I use a little at work but mainly on the Job DSL Plugin OSS project.  At the time of learning Ruby, getting to grips with a dynamic, non-statically-typed language was difficult and exciting.  It also properly introduced me to new concepts which I came to love such as closures and DSLs.  Feeling the grass on the other side of the fence also made me a better Java developer – I became explicitly aware of things that Java did well (it turned out I really like types, for all sorts of reasons) and not so well (there’s a lot of typing for starters, plus a vast amount of syntactic noise, and closures make for much more legible code). 

So now it’s time again to learn another new language, and it’s time to get functional, and my language of choice is Scala.

Functional?

I’ve never done Functional programming before – neither conceptually nor explicitly. My education only properly exposed me to Java, and at the time, that seemed like a lot.  But now, with all the noise coming from the Scala camp, and the fact that it’s rapidly growing JVM language with strong OO credentials and scripting and internal-DSL support built in from the ground up, it seems like it’ll be worth both the mental and time investments. Perhaps the only real benefit is that I become an even more aware of the strengths and limitations of Java itself. That’s no bad thing. Perhaps the only benefit will be that I learn more about the limitations of myself. That’s no bad thing either.

But this vocal community isn’t the only reason I’m starting this now. It seems to me, that with the recent publication of “Atomic Scala”  by Bruce Eckel and Dianne Marsh, that the Scala community is ready to embrace the functional-noobs (read: not bright) amongst us.  How so?  They’ve approached learning the language using a cumulative, step-by-step approach which appeals to me greatly as a student.  They’ve also steered clear of the “there are also 100 other ways of doing this” pitfall which appeals to the “learn via the syntax” rocket scientists, but which to me feels incapable of leading me towards a Functional equivalent of my “OO moment”.

This means they’ve focussed instead on an idiomatic way of doing things, and so there’s a way in for folks like me without a hint of an academic Functional or even Mathematical background. The same thing happens in the Pickaxe Book, and that helped greatly when I did this for Ruby.  Add that to the fact that Thinking in Java was my gateway to my first language, I’m pretty confident that this will be a profitable approach this time.

Full Disclosure

Reading back over the last few paragraphs, despite my alleged humility, it does seem I’m pretty confident of a few things.  Well, that’s because I’ve not been entirely honest.  The book isn’t my first exposure to Scala (or even my second or third) and this won’t even be the first time I’ve read it. I’ve tried to get into Scala before.  Having attended the massively popular JavaONE 2008 session by Martin Odersky, and having heard Dick Wall of the Java Posse enthuse about it, I decided to try and work my way through an O’Reilly Rough Cut of Programming Scala.  But I didn’t get on well – it was the “here’s the syntax” directed approach that I’ve mentioned above, and it felt like I was missing the deeper connections and complete picture I’d need to build of the language in order of it to stick, and for me to properly understand it and be a proficient user of it.

Some more time passed and I was lucky enough to attend numerous Java Posse Roundups and came face-to-face with many hyper-concentrated bursts of enthusiasm in the shape of sessions, lightning talks, hackathons, and tens of “hallway conversations” (there’s not really a hallway to speak of in Crested Butte Church Hall).

Finally, a few years ago, I was even luckier to be asked by Bruce and Dianne if I’d read and comment on an early draft of Atomic Scala.  As long as I could approach it as a complete and utter novice, I was happy, and fortunately, having an idiot on the team was what they were looking for. I was off.  What I read was impressive, but more importantly, made sense to me, and it was that experience which finally drove me to decide to write this blog when the finished book finally came out.

Levels of Ignorance

But before we begin, there’s one more piece of data that must be submitted. Earlier this week I attended TypeSafe’s Fast Track to Scala. I’d finally managed to convince my boss to let me attend this two day course and was super excited that I was getting to attend this two-day immersion.  It started very well, and after day one I felt confident – my head still well above water.  But I’d grown complacent (stupid people do that) and day two plunged me off the edge into a conceptual torret, the bottom of which I had little hope of finding.

Firstly, lets make a few points crystal clear:

• It was not the fault of the course or the trainer that this happened – I failed to pay attention and keep up;
• by the time I realised I’d lost the thread it was too late to ask the relevant catch-up questions – the classroom is not a place to be selfish IMHO

On the train home I did however manage to make some sense at least of where I had lost the thread, and also analyse what I thought were the specific areas which had given me so little that to me felt solid enough to cling on to.  These were:

• Algorithms – I was brought face to face with the fact that my ability to design algorithms is one of my weakest areas. By this I mean low level algorithms. The stuff you learn at University, like bubble sort
• The Functional Idiom – quite simply, in the core of my being, “map” doesn’t mean “apply this to every one of these”, and a “flatmap” is a map laid out nicely

The latter was the thing which slowly undermined me. There came a point, when we were looking at the collections API (which I am excited to eventually learn – it does look very powerful) when I had to keep looking back in the notes to remind me of what all these functions were doing for me.  Sound stupid? Yup.

But it was the former, which was the thing that did for me.  To be frank, I’d been avoiding facing up to this fact for along time and it’s not a Scala-only problem. (See, better Java developer already.) But now here it was, laid out in front of me.  To make it worse I was surrounded by PERL hackers to whom this made perfect sense. I mean, they barely have objects!

Remembering the Kanji

After a good nights sleep, I’d come to terms with my predicament.  I realised I’d been here before - during a short-lived (but successful, until I ran out of time) attempt to learn the 3000+ Japanese Kanji characters with James Heisig’s “Remembering the Kanji”.  What that experience proved was I’m a visual guy – that’s why OO is such an attractive concept for me.  Unfortunately the way that most folks are able to grok algorithms and concepts of functional programming doesn’t play into this, and consequently, few of the learning materials work along non-visual lines.

However, that didn’t mean I was disheartened, but it did mean I’ll need to take things slowly – ensure I fully understand every concept as I move through it. Rather than the “Fast Train (Track) to Scala” this will be more like the “Slow, Stopping, Regional Train to Scala”. But that’s cool, I can handle that.  Armed with Bruce and Dianne’s book, which does work in a way that I’m comfortable with (at least in the draft I read) , I’m going to start out on my journey, and this blog will record my progress.  As I plod slowly on, chapter by chapter, and koan by koan, I’ll put posts up here.  The primary aim will be to help me ensure I fully understand everything, but if its useful for others folks, brill.

Ok, all set? Lets go….