This blog post hopes to convince you that Python is a compiled language. And by “Python”, I don’t mean alternate versions of Python like PyPy, Mypyc, Numba, Cinder, or even Python-like programming languages like Cython, Codon, Mojo1—I mean the regular Python: CPython! The Python that is probably installed on your computer right now. The Python that you got when you searched “python” on Google and downloaded the first thing that came up. The Python that you can pull up just by typing python3 into a fresh command line. That Python. Python is a compiled language.

Some context

Currently, I’m working on material for teaching students how to read and understand programming error messages. We are creating lessons for three programming languages: C, Python, and Java. One of the key points in teaching the nature of error messages is that they are generated at different stages: some error messages are reported at compile-time and others are reported while the program is actively running.

The first lesson was written for C, specifically using the GCC compiler. What these lessons demonstrated was that GCC splits the task of turning your code into a running program into various different stages:

  1. preprocessing
  2. lexical analysis
  3. syntactic analysis
  4. semantic analysis
  5. linking

Furthermore, this lesson discusses the errors that may occur at each stage, and how that will affect the error messages that are presented. Importantly, an error in an earlier stage will prevent errors from being detected at a later stage.

While I was adapting this lesson for Java and Python, I realized that some things would have to change—for example, neither Python or Java have a preprocessor, and “linking” is not really the same thing in either Python or Java, so I just skipped these topics. However, I stumbled across an interesting insight.

Error messages can help you discover compilation stages

The fact that error messages are generated by different stages of the compiler, and compilers generally issue errors from earlier stages before continuing also means that you can discover the stages of your compiler by deliberately creating errors in a program.

So let’s discover the stages of the Python interpreter by playing a little game I like to call…

Which! Is! The! First! Error!

We are going to create a Python program with several errors, each of which attempts to induce a different kind of error message. We know that regular ol’ Python reports only one error message at a time—so the game is which error message will be reported first?

Here is the buggy program:

1 / 0
print() = None
if False
    ñ = "hello

Each line of code generates a different error message:

  1. 1 / 0 will generate ZeroDivisionError: division by zero.
  2. print() = None will generate SyntaxError: cannot assign to function call.
  3. if False will generate SyntaxError: expected ':'.
  4. ñ = "hello will generate SyntaxError: EOL while scanning string literal.

The question is… which will be reported first? If you’re playing along at home, note that, unless otherwise stated, I will be reporting error messages from Python 3.12. Spoilers: the specific version of Python matters (more than I thought it would) so keep that in mind if you see different results.

Round 1

Now that we know the rules and the error messages that we are expecting, let’s start! Without running the code, which of the above error messages will be reported first?

Before I reveal the answer, think about what “interpreted” vs “compiled” language means to you. Here is a terrible Socratic dialogue that will hopefully make you personally reflect on what the difference is:

Socrates: A compiled language is one where the code is first put through a compiler before it is able to be run. An example is the C programming language. To run C code, first you have to run a compiler like gcc or clang, and then finally you can run your code. A compiled language gets converted to machine code—the ones and zeroes that your CPU understands.

Plato: But wait, isn’t Java a compiled language?

Socrates: Yes, Java is a compiled language.

Plato: But isn’t output of the regular Java compiler a .class file. That’s bytecode, isn’t it?

Socrates: That’s correct. Bytecode isn’t machine code, but Java is still a compiled language. This is because there are many problems that the compiler can catch, so you will need to correct errors before your program starts running.

Plato: What about interpreted languages?

Socrates: An interpreted language is one that relies on a separate program, aptly called an interpreter, to actually run your code. An interpreted language does not require the programmer to run a compiler first. Because of this, any errors that you make will be caught while your program is running. Python is an interpreted language—there is no separate compiler, and all errors that you make are caught at runtime.

Plato: If Python is not a compiled language, then why does the standard library include modules called py_compile and compileall?2

Socrates: Well, those modules just convert Python to bytecode. They don’t convert Python to machine code, so Python is still an interpreted language.

Plato: So, both Python and Java are converted to bytecode?

Socrates: Correct.

Plato: Then how is Python an interpreted language and yet Java is a compiled language instead?

Socrates: Because all errors in Python are caught at runtime.

Okay, enough of that. Let’s get to the actual answer. If you run code above in Python 3.12…

🥁🥁🥁🥁

.

.

.

.

.

.

.

.

.

.

You will get this error message:

  File "/private/tmp/round_1.py", line 4
    ñ = "hello  # SyntaxError: EOL while scanning string literal
        ^
SyntaxError: unterminated string literal (detected at line 4)

The first error message detected is on the last line of source code. What this tells us is that Python must read the entire source code file before running the first line of code. If you have a definition in your head of an “interpreted language” that includes “interpreted languages run the code one line at a time”, then I want you to cross that out!

What happened here?

I haven’t done a deep dive into the source code of the CPython interpreter to verify this, but I think the reason that this is the first error detected is because one of the first steps that Python 3.12 does is scanning (also known as lexical analysis). The scanner converts the ENTIRE file into a series of tokens before continuing to the next stage. A missing quotation mark at the end of a string literal is an error that is detected by the scanner—the scanner wants to turn the ENTIRE string into one big token, but it can’t do that until it finds the closing quotation mark. The scanner runs first, before anything else in Python 3.12, hence why this is the first error message.

Let’s close the quotation mark on line 4 to get rid of that error message and try again.

Round 2

With our first error fixed, this is what our code looks like:

1 / 0
print() = None
if False
    ñ = "hello"

There are still errors on lines 1, 2, and 3.

Which! Is! The! First! Error!

Make an educated guess on which will be the first error reported. I’ll spare you the strawman-y Socratic dialogue this time, and get straight to the point.

🥁🥁🥁🥁

.

.

.

.

.

.

.

.

.

.

File "/private/tmp/round_2.py", line 2
    print() = None
    ^^^^^^^
SyntaxError: cannot assign to function call here. Maybe you meant '==' instead of '='?"

It is the SyntaxError on line 2!

What happened here?

Once again, I did not check within the source code of CPython, but I am reasonably certain that the next stage is parsing (also known as syntactic analysis) and the parser reports the first error in the source code. Parsing the whole file happens before running the first line of code which means that Python does not even see the error on line 1 and reports the syntax error on line 2.

At this point, I will point out now that the program I wrote for this exercise is completely nonsensical and there is no right answer for how to fix the error. I have no idea what might be intended by print() = None, so I will fix this by replacing it with print(None) which also doesn’t make sense, but at least it’s syntactically correct.

Round 3

We fixed one syntax error, but there’s still another two errors in our file—one of which is also a syntax error. Here’s what our file looks like:

1 / 0
print(None)
if False
    ñ = "hello"

Which! Is! The! First! Error!

Recall that it seems that the syntax error took priority in round 2. Will it be the same in round 3?

🥁🥁🥁🥁

.

.

.

.

.

.

.

.

.

.

  File "/private/tmp/round_3.py", line 3
    if False
            ^
SyntaxError: expected ':'

Yes! The syntax error on line 3 took priority over the error on line 1.

What happened here

Just as in Round 2, the parser sees the entire file first before proceeding to the next stage. The fix is, as the error message (implicitly) suggests, to insert a colon before the end of the line.

Aside: Differences between Python versions

You might be wondering why I inserted two SyntaxErrors in the same file. Wouldn’t one be enough to show my point?

And here’s where the specific version of Python becomes relevant. If you try the same exercise in Python 3.8 or earlier, the line numbers for Rounds 2 and 3 are swapped!

In Python 3.8, the first error message reported for Round 2 is on line 3:

File "round_2.py", line 3
    if False
            ^
SyntaxError: invalid syntax

And after inserting the missing colon, Python 3.8 reports the following error message on line 2:

  File "round_3.py", line 2
    print() = None
    ^
SyntaxError: cannot assign to function call

Why are Python 3.8 and 3.12 reporting different errors? The reason is that Python 3.9 introduced a new parser. This parser is more capable than the previous, relatively naïve parser. The old parser (an LL(1) for you parsing nerds) was incapable of looking more than one token in advance which means that the parser in-and-of-itself technically accepted Python programs that are syntactically-invalid. Particularly, this limitation prevented the parser to identify whether the left-hand side of an assignment is a valid assignment target. The PEP linked earlier notes that the old parser accepts code that looks like this:

[x for x in y] = [1,2,3]

But this code makes no sense! In fact, the full Python grammar disallows this code. To fix this, a separate, hacky stage used to exist in Python that would check all the assignments and make sure that the left-side is actually something that can be assigned to. This occurred after parsing, hence why the error messages are swapped in the older versions of Python.

Round 4

Okay, with the last syntax error fixed in our program, this is what our program looks like:

1 / 0
print(None)
if False:
    ñ = "hello"

Okay, surely, now the next error message to be reported is the error on the first line. Right? …Right?

🥁🥁🥁🥁

.

.

.

.

.

.

.

.

.

.

Traceback (most recent call last):
  File "/private/tmp/round_4.py", line 1, in <module>
    1 / 0
    ~~^~~
ZeroDivisionError: division by zero

Yes, finally! At last, the error on the first line is reported.

Note also that this error message makes the first appearance of Traceback (most recent call last)—normally a staple of any Python error message, here appearing for the first time in the final round.

What happened here

Python was finally able to start running the code. Once all of the syntax errors are resolved, Python finally is able to interpret the first line, where it is forced to divide a number by zero, raising a runtime error called ZeroDivisionError! We know we’re in “runtime” because Python has printed Traceback (most recent call last) indicating that we have a stack trace. A stack trace is only something that can exist during runtime, which means that this error must have been caught during runtime.

This implies that the errors encountered in rounds 1–3 were… not… runtime errors? Then what were they?

Python is both a compiled and interpreted language

That’s right! The CPython interpreter really is an interpreter. But it also is a compiler. I hope the above exercise has demonstrated that Python must go through a few stages before ever running the first line of code:

  1. scanning
  2. parsing

Older versions of Python added an additional stage:

  1. scanning
  2. parsing
  3. checking for valid assignment targets

Let’s compare this to the stages of compiling a C program from earlier:

  1. preprocessing
  2. lexical analysis (another term for “scanning”)
  3. syntactic analysis (another term for “parsing”)
  4. semantic analysis
  5. linking

Python still performs some compilation stages before running any code. Python really does compile your code first, and just like Java, it compiles to bytecode. The implication for error reporting is that Python has compiler error messages and not every error message in Python occurs during runtime—in fact, only one of the four error messages above was raised at runtime, namely, ZeroDivisionError: division by zero.

You can actually compile all of your Python code beforehand using the compileall module on the command line:

$ python3 -m compileall .

This will place the compiled bytecode of all Python files in the current directory in __pycache__/ and show you any compiler errors. If you want to know what actually is IN that __pycache__/ folder, I did a talk for EdmontonPy which you should check out!

It is only after Python has been compiled to bytecode that the interpreter actually starts.3 I hope that the previous exercise has demonstrated that Python can indeed issue errors before runtime!

“Compiled vs. Interpreted language” is a false dichotomy

It’s a pet peeve of mine whenever a programming language is categorized as either a “compiled” or “interpreted” language. A language is not inherently compiled or interpreted; whether a language is compiled or interpreted (or both!) is an implementation detail.

And I’m not the only one that thinks like this. Laurie Tratt has a wonderful article arguing this exact same point by writing an interpreter that gradually becomes an optimizing compiler. It’s really neat!

Another fantastic resource is Bob Nystrom’s Crafting Interpreters. Here are some quotes from chapter 2:

What’s the difference between a compiler and an interpreter?

It turns out this is like asking the difference between a fruit and a vegetable. That seems like a binary either-or choice, but actually “fruit” is a botanical term and “vegetable” is culinary. One does not strictly imply the negation of the other. There are fruits that aren’t vegetables (apples) and vegetables that aren’t fruits (carrots), but also edible plants that are both fruits and vegetables, like tomatoes.

Fun fact: this was when first I learned what the deal is with tomatoes. In cooking, they’re a vegetable. And they’re, botanically, a fruit. They’re both. They can be both, and that’s okay. Wait, weren’t we talking about Python? Bob, bring us back:

When you run your Python program using [CPython], the code is parsed and converted to an internal bytecode format, which is then executed inside the VM. From the user’s perspective, this is clearly an interpreter—they run their program from source. But if you look under CPython’s scaly skin, you’ll see that there is definitely some compiling going on.

The answer is that it is both. CPython is an interpreter, and it has a compiler.

So why does it matter? Why is it counterproductive to make a hard distinction between “compiled” and “interpreted” languages?

“Compiled vs. Interpreted” limits what we think is possible with programming languages

A programming language doesn’t have to be defined by whether it’s compiled or interpreted! And thinking in such a rigid way limits what we believe a given programming language can do.

For instance, JavaScript is commonly lumped into the “interpreted language” category. But for a while, JavaScript running in Google Chrome would never be interpreted—instead, JavaScript was compiled directly to machine code!4 As a result, JavaScript can keep pace with C++. For this reason, I am really tired of arguments that say that interpreted languages are necessarily slow—performance is multifaceted and depends on much more than the “default” programming language implementation. JavaScript is fast now. Ruby is fast now. Lua has been fast for a while.

What about programming languages that are typically labelled as compiled?

You wouldn't interpret a C program

Do you want a C interpreter? Here you go! Now you can write shell scripts that feature undefined behaviour!

REPLs for everybody!

One of the ways that I fell in love with programming was sitting in front of the REPL of languages like Python and Basic. That immediacy is what made programming thrilling for me, and is why I often reach for REPLs in my day-to-day life. But it seems like you can only have an interactive prompt for interpreted languages—so you can’t have a REPL for compiled languages, right?

Consider the Glasgow Haskell Compiler (GHC). One of the ways that I learned Haskell (and promptly forgot it 😉) is by messing around in the interactive console. But, as the name suggests, GHC really does compile Haskell code.

This makes being a “compiled” language no excuse for lacking an interactive interpreter! Do you want an interactive Java experience, similar to Python? It turns out that jshell has been bundled with Java since 2017.

But all of this is a distraction from the real distinction that we should be teaching students:

The true distinction: static vs. dynamic

The true distinction that we should be teaching students is the difference between properties of languages that can be determined statically—that is, by just staring at the code without running it—and properties that can only be known dynamically, during runtime.

Notice that I said “properties” and not “languages”. Every programming language chooses its own set of properties that can be determined either statically or dynamically, and taken together, this makes a language more “dynamic” or more “static”. Static versus dynamic is a spectrum, and yes, Python falls on the more dynamic end of the spectrum. A language like Java has far more static features than Python, but even Java includes things like reflection, which is inarguably a dynamic feature.

I find it understandable that dynamic vs. static is often conflated with compiled vs. interpreted—usually, languages that use interpreters have more dynamic features, like Python, Ruby, and JavaScript. Languages with more static features tend to be implemented without interpreters, like C++ and Rust. Then there’s Java that is somewhere in the middle.

Static type annotations in Python have been gradually (hehe) gaining adoption in codebases, and one of the expectations is that this can unlock performance benefits in Python code due to more things being known statically. Unfortunately, the truth is not that simple. It turns out that both types in Python (yes, just types in general—consider, metaclasses) and annotations themselves are dynamic features of Python, which makes static typing not the boon for performance that you would expect.

Conclusion

It doesn’t matter whether Python is compiled or interpreted. What matters is the set of properties that can be determined statically (that is, before runtime) in Python is relatively small, which means more errors tend to make themselves apparent at runtime than in programming languages that have more static features. This is the actual distinction that matters, and it is a far more granular and nuanced distinction than “compiled” vs “interpreted”. For this reason, I think it is important to emphasize the particular static and dynamic features when rather than teaching the tired distinction between “interpreted” and “compiled” languages.

EDIT: For source code for all the rounds (including a bonus round!) check out my GitHub repository. You can also see the results for errors across different implementations of Python, check out the different jobs in GitHub Actions.

  1. I did not realize just how many Python compilers/Python-like languages exist. And this is very much a non-comprehensive list. ↩

  2. Nobody would ever say this. ↩

  3. Note that importing a module in Python happens at runtime, which means that any syntax errors in an imported module will give you error messages during runtime. You can still check for syntax errors in imported modules beforehand by using the compileall module. ↩

  4. From what I understand, an interpreter called Ignition was added in 2016 to the V8 JavaScript engine, because the old “full-codegen” ate up a giant amount of memory. ↩