One million ought to be enough for anybody

Programming languages generally have limits—explicit or implicit—on various aspects of their operation. Things like the maximum length of an identifier or the range of values that a variable can store are fairly obvious examples, but there are others, many of which are unspecified by the language designers and come about from various implementations of the language. That ambiguity has consequences, so nailing down a wide variety of limits in Python is the target of an ongoing discussion on the python-dev mailing list.

Mark Shannon posted a proposal "to impose a limit of one million on various aspects of Python programs, such as the lines of code per module". One million may seem like an arbitrary number (and it is), but part of his thinking is that it is also memorable, so that programmers don't need to consult a reference when wondering about a language-imposed limit. In addition, though, certain values stored by the Python virtual machine (e.g. line numbers) are 32-bit values, which obviously imposes its own limit—but one that may be wasting space for the vast majority of Python programs that never even get close. Beyond that, overflowing those 32-bit values could lead to security and other types of problems.

As Shannon pointed out, a range of -1,000,000 to 1,000,000 could fit in 21 bits and that three of those values could be packed into a 64-bit word. "Memory access is usually a limiting factor in the performance of modern CPUs. Better packing of data structures enhances locality and reduces memory [bandwidth], at a modest increase in ALU usage (for shifting and masking)." He suggested that the data structures for stack-frame objects, code objects, and objects themselves could benefit from packing in that fashion. "There is also the potential for a more efficient instruction format, speeding up interpreter dispatch."

He proposed using the limit for seven different aspects of Python programs:

He also addressed the "Isn't this '640K ought to be enough for anybody' again?" question, which is something that immediately comes to mind when arbitrary limits are proposed. He noted that the Java virtual machine (JVM) limits many program elements to 65535 (which fits in 16 bits); that can be constricting, but mostly for program generators rather than hand-written code. The limit he is proposing for Python is a good deal larger than that and he believes it would not prove to be a real impediment to human-generated code. "While it is possible that generated code could exceed the limit, it is easy for a code generator to modify its output to conform."

Reactions

He provided short justifications for some of the limits, but many of those who commented in the thread were concerned with how much benefit they would actually provide. Chris Angelico asked whether Shannon had done any measurements to see how large of an effect there would be. Steven D'Aprano agreed with the premise of getting memory and security benefits, but thought it was "a bit much to expect" Python developers to take the anticipated speed increase on faith. Steve Dower thought the overall idea was not unreasonable, though he had some concerns as well:

Dower is referring to the Windows MAX_PATH value, which restricts path names in the Win32 API to a maximum of 260 characters. He thought several of the proposed limits did seem reasonable, though the "lines per module" limit "feels the most arbitrary". Comments and blank lines would certainly count against the limit, which gave him pause. The "classes in a running interpreter" limit is a bit worrisome, he said, but there may be ways to deal with programs that go beyond it while still getting any gains it brings: "The benefits seem worthwhile here even without the rest of the PEP."

But Rhodri James thought that limits like those proposed will eventually become a problem for some; he also objected to the idea of packing the counts into smaller bit widths due to the inefficiency of masking and shifting on each access. Gregory P. Smith was, in general, in favor of limits, but was concerned that code generation would run afoul of them; he noted that the JVM limits have been a big problem in the Android world. Others in the thread also pointed to the JVM limits as being problematic.

Guido van Rossum posted some thoughts on the idea. He wondered a bit about the problem being solved and expressed some skepticism that, for example, representing line numbers in 20 bits rather than 32 is really going to be much of an efficiency gain. He was concerned about "existing (accidental) limits that caused problems for generated code". But he also pointed out that the existing CPython parser is limited to 100 levels of nested parentheses (and likely nested indent levels as well) and there have been no complaints that he has heard.

A real world example of a file with one million lines was noted by Oscar Benjamin. It is a test file for SymPy that crashed the Python 3.6 interpreter (though that was fixed in 3.7.1). The actual test is only around 3000 long lines, but it gets rewritten by the pytest framework into a file with more than one million lines. Benjamin also pointed out that a limit of one million bytecode instructions is even more restrictive.

A distinction should be made between limits for the language itself and for those of the CPython implementation, Jim J. Jewett said. He noted that "there is great value in documenting the limits that CPython in particular currently chooses to enforce", but that making the limits too high for the language itself would, for example, potentially leave out implementations like MicroPython.

PEP 611

Shannon took in the feedback and reflected it in a PEP 611 ("The one million limit"). That led to another thread, where there were more calls for some kind of benchmarking for the changes in order to reasonably evaluate them. Barry Warsaw also said that "there is a lack of clarity as to whether you are proposing a Python-the-language limit or a CPython-the-implementation limit". If the limits are for the language "then the PEP needs to state that, and feedback from other implementation developers should be requested".

A few days later, Shannon asked that commenters be more precise about their concerns. There are costs associated with either choice, but simply stating a preference for a higher limit is not entirely helpful feedback:

Angelico again asked for numbers, but Shannon said that the performance increase is difficult to quantify: "Given there is an infinite number of potential optimizations that it would enable, it is a bit hard to put a number on it :)". As might be expected, that hand waving was not entirely popular. But part of the problem may be that Shannon sees the potential optimizations partly as just a byproduct of the other advantages the limits would bring; as Nathaniel Smith put it:

Justification is certainly needed for making changes of this sort, Shannon said, but thought that the current limits (or lack thereof) came about due to "historical accident and/or implementation details", which would seemingly allow a weaker justification. Van Rossum took issue with that assertion:

Shannon replied that it may be part of the philosophy of the language, "but in reality Python has lots of limits." For example, he pointed out that having more than 2³¹ instructions in a code object will crash CPython currently; that is a bug that could be fixed, but those kinds of problems can be hard to test for and find.

The limits on the number of classes and on the number of coroutines were specifically raised as problematic by Van Rossum. Changing the object header, which is one thing that a limit on classes would allow, is a change to the C API, so he thinks it should be debated separately. A coroutine is "just another Python object and has no operating resources associated with it", so he did not understand why they were being specifically targeted. Others agreed about coroutines and offered up suggestions of applications that might have a need for more than one million. Those objections led Shannon to drop coroutines from the PEP as "the justification for limiting coroutines is probably the weakest".

Steering council thoughts

The Python steering council weighed in (perhaps in one of its last official actions, as a new council was elected on December 16) on the PEP. Warsaw said that it had been discussed at the meeting on December 10. The council suggested that the PEP be broken up into two parts, one that applies to all implementations of the language and would provide ways to determine the limits at runtime, and another that is implementation-specific for CPython with limits for that implementation. In addition: "We encourage the PEP authors and proponents to gather actual performance data that can be used to help us evaluate whether this PEP is a good idea or not."

Shannon is still skeptical that using "a handful of optimizations" to judge the PEP is the right approach. It is impossible to put numbers on all of the infinite possible optimizations, but it is also "impossible to perfectly predict what limits might restrict possible future applications". However, he did agree that coming up with example optimizations and applications that would suffer from the limitations would be useful.

In yet another thread, Shannon asked for the feedback, with specifics, to continue. He also wondered if his idea of choosing a single number as a limit, mostly as a memory aid, was important. Several thought that it was not reasonable to pick a single number for a variety of reasons. As D'Aprano put it, no one will need to memorize the limits since they should be rarely hit anyway and there should be a way to get them at runtime. Beyond that, there are already limits on things like nested parentheses, recursion depth, and so on that are not now and are not going to be one million.

Paul Moore agreed that a single limit value was not important, though he was in favor of choosing round numbers for any limits, rather than something based on the implementation details. He also may have summed up how most are thinking about the idea:

That's where things stand at this point. It would seem there is interest on the part of the steering council, given that it took up the PEP in its early days, though the council's interest may not entirely align with Shannon's. Ill-defined limits, with unclear semantics on what happens if they are exceeded, would seem to have little benefit for anyone, however. Some tightening of that specification for CPython and additional APIs for the language as a whole would be a great step. It might allow those interested to tweak the values in an experimental fork of CPython to test some optimization possibilities as well.