Why I'm excited about lazy

A new version of Python dropped this month, with a feature I have been fiending for:

The 'lazy' keyword.

Because Python normally tries to slurp in every imported module, right away, before starting the first line of real program code. Which often is not a big deal. But sometimes it makes the UX sluggish, or cause other performance problems.

Look at this little program, for example:

# demo.py
import pandas
import argparse

parser = argparse.ArgumentParser()
parser.add_argument(
    'sourcefile',
    help='Data source file CSV')

class Something:
    def foo(self):
        pass # do something with pandas

if __name__ == '__main__':
    args = parser.parse_args()
    # some code goes here

The UX problem is with that first line of code:

import pandas

Because if you have worked with Pandas before, you know that single import statement can take a while. Much longer than a normal module import, because of all the heavyweight code it has to pull in.

As a result, if you run python demo.py --help, where you just want to see the docs for the command-line interface, and not actually run the program... your experience is:

  1. Type python demo.py --help and press enter
  2. Wait longer than you want, for no good reason
  3. See the help output

That delay really is needless. You're not actually using Pandas at all here; you just want to get the help message, not crunch a dataframe.

Before now, you'd have to resort to some uncivilized hack in order to get rid of step 2; like moving the import statement into weird place, or more gruesome and shameful tactics I cannot bring myself to even tell you about.

All of which are gracefully avoided by prepending one word to that line:

lazy import pandas

Now that import is magical. It does NOT drop everything and and wait to import the full gargantuan machinery of Pandas into the process, before going to the next line.

Instead, 'pandas' becomes a kind of reference... a proxy to the actual module. So that if and when you start slinging dataframes, that module is loaded at the last moment, just in time.

The performance difference is real:

% /usr/bin/time python3.15 demo.py --help
usage: demo.py [-h] sourcefile

positional arguments:
  sourcefile  Data source file CSV

options:
  -h, --help  show this help message and exit
        0.27 real         0.23 user         0.03 sys

Compared to the lazy version:

% /usr/bin/time python3.15 demo-with-lazy.py --help
usage: demo-with-lazy.py [-h] sourcefile

positional arguments:
  sourcefile  Data source file CSV

options:
  -h, --help  show this help message and exit
        0.05 real         0.03 user         0.01 sys

That's 270 milliseconds versus 50 milliseconds. A hyooge 5.4x speedup.

Even though I usually talk about higher-level coding strategy, rather than details of syntax like this...

I think it serves as a parable in our quest to be the best engineers we can be. Several parables, in fact.

One is that it helps us think about how lazy-loading of other resources can improve performance, responsiveness and UX. Whether that's in a small command-line program, or some resource in a vast distributed architecture.

Another is that we can evaluate the quality of a coding agent, based on whether we see it taking opportunities like this in the code it generates.

This lazy situation came up in the Boosted Code group discussion yesterday. Among other fun topics, like my two categories of coding agents; how to use the BC curriculum as an engineering manager; and mucho more.

If you are not in Boosted Code yet... well, too bad you missed it.

But you can be at the next one. Learn about Boosted Code here:

About Boosted Code