Announce: JustHub Sherkin Release

June 13, 2012

I am pleased to announce the Sherkin release of the JustHub distribution of Haskell for Enterprise Linux and the release of the Hub source code on which the distribution is based to Hackage and GitHub.

The JustHub distribution is distinctive in providing a comprehensive back-catalogue of GHC tool chains (back to 6.10.4) and major Haskell Platform releases (currently back to 2011.2.0.1) as well as the timely appearance of leading edge GHC releases (such as GHC 7.4.2) in addition to the Haskell Platform de jour (2012.2.0.0), all of which can be deployed simultaneously on the same system.

Previous releases of the distribution have been confined to Red Hat Enterprise Linux versions 5 and 6 (and of course CentOS and Scientific Linux); this release expands this coverage to Fedora Linux 16 and 17, which can deployed along side or integrated into Fedora Linux’s excellent native Haskell distribution.

The JustHub distribution is based on the Hub system for sandbox development that allows each work tree to work in its own sandboxed environment with a specific tool chain and a private user-package database. All of the standard Haskell tools inter-operate cleanly and transparently with the sandboxes and operate in the normal way outside of them.

Sandboxed environments (hubs) can be shared between work trees as well as being (re)named, annotated, replicated, swapped, archived, locked and removed. Proper package deletion with the option of garbage collecting orphaned code and documentation is also supported.

If anyone would like to incorporate the hub system into a Haskell distribution or port it to a non-Posix system then they are very welcome to contact me (though be warned: time is tight just now).

Note that while the distro has had plenty of serious usage (funnily enough it is popular with build-bot admins) it has also been recently extended and revised – the software remains in beta testing for now.

For news, documentation, articles, download and installation instructions, links to the repositories and trackers etc., see http://justhub.org.


Experiments in Haskell Packaging

January 2, 2012

[This article was posted on the Haskell Cafe mailing list today.]

The http://justhub.org Haskell distribution for Enterprize Linux (RHEL/CentOS 5 & 6) is now live.

The distribution deviates from current practice. When you upgrade to a new Platform from justhub.org you add the new platform to your Haskell infrastructure rather than replacing the old platform with a new one.

Why?

  • Stability: Enterprise Linux places a strong emphasis on stability and updating packages should not break things. (This has provided some challenges for us as RHEL 5 is still using GCC  4.1.2 and Binutils 2.17.50.)
  • Flexibility: Different projects will in general need to work with different versions of the platform and toolkits and upgrade according to their own schedules. I believe Haskell’s strong-typing sharpens the need for this. Production projects will need to upgrade according to their own schedules while other projects will want to stay up-to-date. In general when you check out a source tree you want to build with the tools and libraries that it has been tested with and move on according to the project’s timetable.
  • Sandboxes: The same mechanism for managing the multiple toolkits and platforms can be adapted to provide an integrated sandbox utility. The logic that dictates that projects have different tool-chain requirements applies to the user packages too. (‘Handbox’ doesn’t really work so we use Hubs instead.)
  • Package Deletion: The infrastructure can also be expanded to complete the package management tools with package removal and garbage collection of library code.
  • The Catalogue: All of the GHC releases can be added as they become available.

In general the distribution works just like any other – after an upgrade invoking ‘ghc’ without any other configuration will invoke the compiler for that latest platform, which should behave as it would on any other (working) distribution.

But after an upgrade:

  • any projects configured to work with specific tool chains and libraries will continue undisturbed;
  • you can create a new hub based on the new platform and experiment with it on newly-checked-out source tree without disturbing an existing working tree (or just swap the hub into an existing work tree and back out again if things don’t work out);
  • even on release of a new version of a package (or if you want to experiment with your existing packages) you can easily duplicate an existing hub and delete, upgrade and add packages on a topic branch.

Of course ‘serious developers’ have been doing this with home-brew infrastructure for ever. Nevertheless I have found doing it with well-integrated tools to be quite liberating.

It might also help new users to make the transition to serious developers more quickly.

Thoughts anyone?


Haskell for EL Beta Release

January 1, 2012

Haskell for Enterprise Linux was released on the stroke of the new year.

Here are the highlights:

Share and Enjoy


Enterprise Linux for Haskell

October 25, 2011

It looks as if my Haskell for Enterprise Linux is set for wider circulation. Unfortunately I have something that will keep maxed out for a couple of weeks while the best explanation of what I am trying to do with the release went into a ‘heads up’ email to Simon Marlow, Simon Peyton Jones, Ian Linagh, Duncan Couts and Don Stewart. So here is a lightly edited copy of it.

Share and Enjoy

***

Folks,

As I think most of you know I have been threatening to make a RHEL/CentOS Haskell distribution for some time. Well for the last few months I have been making good and I think I am ready for an alpha release.

The release is a little different from the other one I am aware of, so I want to explain what I am doing here first.

Stability is King for Enterprise

Especially for an Enterprise Linux release I felt that we needed something that allows multiple releases to sit together. For my own production servers I can’t swap over the platform to fit in with distro release cycles. Because the servers use CentOS this has never been an issue, but any distribution that did delete old tools and platforms would not be useful for my day job, except perhaps as a handy source of bootstrap compilers!

I have found that even my own crafted setups are surprisingly cumbersome to work with, especially since I have gotten more heavily into the compiler-building game, but even while working on the client code base I have noticed a marked tendency for me to stick to well-proven collection of packages (especially after the an accidental cabal update destabilised my setup at an unwelcome time – after which I made a careful note of the exact collection of packages I was using).

So, for the EL distribution I have tried to make it easier to smoothly integrate multiple releases at the system level and manage multiple build contexts at the user level, all using a simple device that organises the excellent mechanisms provided by GHC and Cabal in a way that could prove helpful to the user.

From 6.10.4 to 7.2.1

So this distribution provides many platforms and compilers, all of which can reside on the same system.  The repository contains the three major, stable platforms to date (2011.2.0.1, 2010.2.0.0, 2009.2.0.2, though for the moment without Haddock and without OpenGL on the older platforms) and every publically-released GHC since 6.10.4.

How does it work? If you install the Haskell package then you will get the platform du jour (2011.2.0.1) and updating the package will download and install later platforms as they become available. Old platforms won’t go away after a mere update (but it could be manually erased), and any source trees that so configured will continue to use their designated platform.

If older compilers and platforms are needed from the get go then the individual packages can be installed.

The GNU Tool and 5.5G

Because I have noticed some problems with the old-ish gcc (4.1.2) and binutils (2.17) supplied with RHEL/CentOS 5 the distribution comes with its own inboard gcc (4.6.1) and binutils (2.21). RHEL/CentOS 6 use 4.4.2 and 2.20, which I suspect I could use for the Haskell release for these O/Ss (which I haven’t prepared yet), but I think I will provide the same approach and GNU tools on EL6 as EL5, and prioritise stability in the HP across both O/Ss. The cost is 72M+19M on download, 147M+76M unpacked. In total, it’s 200M for the first full platform and 109M for each extra tool chain and 25M for each of the platform libraries (12M without OpenGL). 1.2G of disc space is needed to install haskell, rising to 5.5G for the works.

So how does it really work?

Out of the box, if you type ghc or cabal you will get the default platform and the tools and they should work exactly as they would anywhere else.

If you want to access a different tool chain or fix the tool chain that you get (or you users get by default) you have options. Staying with the user options, you may indicate through the HUB environment variable the platform or compiler you desire like so:

$ HUB=2010.2.0.0 ghc-pkg list

Or you can run a little tool that will drop a marker in your current directory. Provided the HUB environment variable is not set, then any tools run from this or any of its sub-directories (as defined by getCurrentDirectory) will use the specified platform and tool chain.

$ cd ~/brave-new-world

$ hub set 7.2.1

In these directories it is just as if you are on a system with 7.2.1 installed in /usr/bin.

Obviously /usr/bin contains a set of links from the tools that we know and love to the hub program which looks at the configuration files dropped by the distributions, the HUB environment variable and/or the markers in the current and parent directories and invokes the relevant tool (with HUB set to ensure consistent usage regardless of the changes in directory – important for cabal install).

If the user wishes, this trip through the hub can do some more work and provide cabal-dev type functionality. Until now we have been working with system hubs – those that came with the compiler and platform packages, but the users can create their own hubs with package databases attached…

# create user hub ‘bnw’ based on GHC 7.2.1

$ hub init 7.2.1 bnw

$ hub info bnw

bnw (user hub)
  GHC              : 7.2.1
  Tools            : /usr/hs/ghc/7.2.1/bin
  Cabal            : /usr/hs/cabal/cabal
  Package DBs
  global           : /usr/hs/db/7.2.1.d
  user             : /home/apache/.hubrc/lib/bnw/package.config

# and a user hub based on the H.P. 2011.2.0.1

$ hub init 2011.2.0.1 tt

$ hub info tt

tt (user hub)
  GHC              : 7.0.3
  Haskell Platform : 2011.2.0.1
  Tools            : /usr/hs/ghc/7.0.3/bin
  Platform Tools   : /usr/hs/hp/2011.2.0.1/bin
  Cabal            : /usr/hs/cabal/cabal
  Package DBs
  global           : /usr/hs/db/2011.2.0.1.d
  user             : /home/apache/.hubrc/lib/tt/package.config

Now you can install packages into the hub (with cabal), copy, rename and remove hubs (with hub), analyse the packages (with ghc-pkg), etc.

And How is it has working in practice?

I have found this to be quite liberating. My standard smoke test for the repository is to install everything and run this script to prepare a hub for each platform and compiler capable of compiling the hub programme itself, building the compiler and running a simple test to make sure that the executable is functioning. I started with 7.0.3, my natural base. Going forward into 7.2.1 I was browbeaten by the tools into making my code 2010 clean. This turned out to be quite a quick and easy task. Because the tools were to hand and made it really broke down my tendency to stick with the tried and trusted.

Going the other way I discovered some ‘missing’ packages in the 2010 platform.

# we need the platform

$ sudo yum install haskell-platform-2010.2.0.0-hub

# now create a new hub based on the 2010 platform

$ hub init 2010.2.0.0 hub-2010

# and switch the directory to the new hub

$ hub set hub-2010

# install what we need and rebuild

$ cabal install hexpat

$ make clean all

# oops – whining about missing Text.Regex

$ ghc-pkg list

$ HUB=hub-2011 ghc-pkg list

$ HUB=hub-2011 ghc-pkg describe regex-compat

# hmn, missing regex-compat

$ cabal install regex-compat

$ make

# we are good to go

When it came to 2009.2.0.2 I ran into problems getting hexpat to work. Essentially type errors on 6.10.4 pushed me back to a version of hexpat that would have required modifying the hub source code. Now I have no substantive reason for making hub 6.10.4-friendly so I stopped at 6.12.1, but it was a relatively pleasant exercise.

Perhaps contrary to expectation, by making it easy for people to hold onto their old compilers while acquiring the latest tools, we could see the wider code base tracking the tools more closely.

Cor blimey this email [post] is getting a bit long!

Naturally it is all a little rough, but I think still quite usable. If there are any flaws in the scheme I think you folks will be quick to point it out, which is why I wanted to run it past you before take it wider.

If you would like to try it out the instructions should be complete. (Let me know if they are not!)

Cheers,

Chris


Hard Time Learning Haskell?

July 13, 2011

Last week Ryan asked this excellent question:

For every single other programming language I’ve learned in the past, I was able to make useful, working programs in at most a week after learning about the language. But not for Haskell. Yes I understand what types and type classes are, how to create my own functions and I have a decent grasp of the syntax, but I’m just not sure where to begin. When I want to make something useful, I have to stick everything into a “do” statement and then afterwards I just get stuck and don’t know how to proceed.

How would you guys recommend I solve this problem?

Of course Haskell follows in a radical tradition of establishing platonic functiontional/applicative/declarative core language free of grubby earthly concerns. The machinery provided by the core is then used to build the I/O.

Note that tools like GHC aren’t built in this way but have been planned, constructed and refined with the concerns of real-world applications. But this all takes place within the functional framework established by the core. (A prime motivation for this radical dicipline [if 1980s research grant applications are any guide] was the opportunities it would open up for exploiting massively parallel architectures — see, for example this talk by Simon Peyton Jones).

So most Haskell introductory texts wait until about the penultimate chapter before tackling I/O. But this does not at all jibe with the conventional evolution in learning a programming language where you expect to be writing ‘real programmes’ within a week.

My advice therefore is to kick back, spend some time on (e.g.) the first 17 chapters of Thompson’s book. You should definitely be expecting to spend (at least) the first week interacting with your scripts though the ghci evaluation prompt.

But can Haskell I/O be tackeld in week 2?


Just Haskell

July 10, 2011

Welcome to Just Haskell, a blog on the Haskell programming language.

What’s in a Name?

The name of the blog is a play on several things, but now would be an appropriate time to talk about beginnings and the beginnings of the Haskell language. The most important innovation of Haskell is its monadic I/O system. It wasn’t there at the start, but invented and added subsequently (in 1996) after Philip Wadler’s early nineties papers sparked the revolution. In his ’92 POPL paper Wadler mentioned (among others) Michael Spivey’s work on monadic exception handling with the Maybe type as an important precursor. Thus I justify the cheeky name of my blog — a tribute to innovation that Haskell and its tools were intended to encourage.

While we are on beginnings I would like to acknowledge two personal pre-Haskell influences. I came to Haskell and functional programming through the Miranda programming language, David Turner’s 1986 Sigplan notices paper being especially influential. This compactness and elegance was on a different plane from anything else I had encountered. I still like to write out the definition of the prime numbers from time to time (in Haskell of course):

primes :: [Integer]
primes = sieve [2..]
      where
        sieve (p:xs) = p : sieve [ x | x<-xs, x `mod` p /= 0 ]

This has got to be one for the book.

For me the key innovation of Monadic I/O was to apply type abstraction to the data structures (streams and continuations) in such a way that made functional programs performing I/O composable. Simon Thompson's 1987 report Interactive functional programs: a method and a formal semantics was the first place I encountered this idea and why it was so important. When the Monads papers started emerging it seemed like a very good thing indeed.

Enough already. Time to blog.


Follow

Get every new post delivered to your Inbox.