Adoption and Python Compatibility
When I started generating Python, I only thought of it as a stepping stone towards generating machine code or C. But over time, I realized the ability to generate Python is valuable in itself, just like Typescript does for Javascript. The question is, what degree of compatibility should I aim for?
Similar Enough to Understand
If I want the generated code to look like normal Python code, the input language cannot be too different from Python. I’m not sure how different Newton can be before I lose the ability to translate it to idiomatic Python, but there is a limit.
Regardless, I want people to be able to move between Newton and Python easily, so the languages must be quite similar for that reason. For instance, the built-in collections (list, dict, and set) must have essentially the same interface.
Using the same method names and syntax will go a long way, but there are more subtle considerations such as the iteration order. Python dictionaries preserve the insertion order, so the Python target will inherit that behavior without effort on my part. But that means the Dict implementation for the C target must preserve the order too, since all targets should have the same semantics. For the same reason, both targets must use bignums instead of limiting integers to 64 bits.
Some things will have to be different, though, since I still want to be able to generate efficient C. Static typing is a must for performance, and that means heterogenous collections have to go. Anything fundamentally incompatible with Hindley Milner type systems is out.
Different Enough to Matter
There have been many attempts to improve Python’s performance over the years, for instance Python compilers such as Codon and Nuitka, and supersets like Cython and Mojo.
A common reaction to these efforts is: “If it can’t run my existing code unchanged, I’m not interested”.
Striving for full compatibility makes it much harder to generate efficient code, so I have to reject that idea. But it would be a mistake to assume the next best thing is 99% compatibility. That still would not be good enough to satisfy the nay sayers, and minimizing the differences to that degree would prevent me from removing the warts as well.
I believe breaking more things can actually increase adoption–The benefits just have to be big enough to offset the cost and get people’s attention.
My criteria for breaking compatibility will thus be whether it is:
- Necessary to achieve my other goals, or
- Addressing something that is widely regarded a design flaw.
Old Batteries
So much for the core language. When it comes to the wider eco system, a lot of things can simply be left for whoever needs it to port on their own, maybe with the help of a translation tool.
“Batteries included” was a major selling point 20 years ago, but I think it matters less today. Some of the batteries have even become liabilities.
Stefan