First Milestone Reached

I am now able to compile small but useful Newton programs that exercise the full feature set to Python! This is a major breakthrough because until now there were always things that worked for simple cases, but would break when you tried something more complex.

Definition of Done

My criteria for reaching the milestone was that I should be able to compile a program that counts words in a string. Sounds easy, but there is actually a ton of things that have to work together:

  • Type inference / type checking
  • For loops
  • Generators
  • Importing other modules
  • A standard library with collections and utility functions
  • Sum types, such as Bool and Maybe
  • Record types
  • Comprehensions
  • Polymorphic functions
  • Overloading
  • Pattern matching

The Test Case

Here it is, the piece of land I conquered. All of the above features are being used, but some of them in other modules or hidden behind syntactic sugar (such as the for loop).

def word_freq(text):
    let res = dict()
    let words = [lower(word) for word in split(text, " ") if isalpha(word)]

    for word in words:
        if word in res:
            res[word] = res[word] + 1
        else:
            res[word] = 1

    return res

def main():
    let freq = word_freq("foo 123  BING bar BING bing BInG   Bing  ")
    print(freq)

main()

The output is:

{foo: 1, bing: 5, bar: 1}

If the language was more complete I would spend a minute to format the result better, but formatting strings is still painful. No f-strings, and print cannot take multiple arguments yet.

Comparison to Python

As you can see it looks a lot like Python, except for the let keyword and that it’s using function calls instead of method calls everywhere (the latter will change).

Despite the absence of type annotations, the example is statically typed since the type of every expression can be inferred.

Generated Python

I am not going to include the generated code here, because it’s just a straight-forward translation of the intermediate representation and doesn’t look like idiomatic Python. I don’t want people to get the wrong idea and think the current output is what they will get.

I explained how generators and match statements are translated in previous blog posts, and that is still how I’m doing it. There is a small number of predefined generators for iterating over collections and ranges, but creating new generators is very cumbersome since the yield keyword isn’t supported yet and you have to perform closure conversion by hand.

Stefan