Handling argc==0 in the kernel [LWN.net]

Handling argc==0 in the kernel

Posted Jan 28, 2022 15:47 UTC (Fri) by larkey (guest, #104463)
Parent article: Handling argc==0 in the kernel

I think there are few things confused here and in the Kernel ML, namely the case a) argv == NULL and the case b) argv[0] == NULL:

> If the argv array passed to execve() is empty (or the argv pointer is simply NULL), the first pointer in the new program's argv array will be NULL, and the envp array will start immediately thereafter.

POSIX says that argv == NULL isn't allowed:

> The argument argv is an array of character pointers to null-terminated strings. The application shall ensure that the last member of this array is a null pointer.

(https://pubs.opengroup.org/onlinepubs/9699919799.2018edit...)

That's also what the kernel bug (https://bugzilla.kernel.org/show_bug.cgi?id=8408) from years back is about. It is *not* about enforcing argc >= 1 (i.e., argv[0] != NULL).

The patch by Ariadne Conill does *not* address what was reported in the bug, despite claiming so. It enforces that case b) never happens which is a stricter requirement and indeed not required by POSIX, as correctly pointed out by Heikki Kallasjoki and Rich Felker. This is just a recommendation:

> The value in argv[0] should point to a filename string that is
associated with the process [...]

I agree that Linux should not allow argv == NULL, that's definitely against the spec. argv[0] == NULL is a different matter although I think it is a reasonable demand, from a security stand point.

-------

Appendix:

POSIX on shall and should:

> shall
>
> For an implementation that conforms to POSIX.1-2017, describes a feature or behavior that is mandatory. An application can rely on the existence of the feature or behavior.

> should
>
> For an implementation that conforms to POSIX.1-2017, describes a feature or behavior that is recommended but not mandatory. An application should not rely on the existence of the feature or behavior. An application that relies on such a feature or behavior cannot be assured to be portable across conforming implementations.

(https://pubs.opengroup.org/onlinepubs/9699919799.2018edit...)

to post comments

Handling argc==0 in the kernel

Posted Jan 28, 2022 16:26 UTC (Fri) by matthias (subscriber, #94967) [Link] (26 responses)

> I think there are few things confused here and in the Kernel ML, namely the case a) argv == NULL and the case b) argv[0] == NULL:

Probably the confusion is for a simple reason. The two things are more or less the same. If the execve() function is called with argv == NULL, the kernel actually constructs a NULL-terminated array argv with argv[0] == NULL and calls main(argv,0). Thus for the called application, these two things are indistinguishable.

> I agree that Linux should not allow argv == NULL, that's definitely against the spec.

From the examples in the discussions it seems that the "lazily written tests" do exactly this: call execve() with argv == NULL. If Linux disallows this, these tests will break due to an ABI/API change.

> argv[0] == NULL is a different matter although I think it is a reasonable demand, from a security stand point.

Yeah, but disallowing the former and allowing the latter does not make any sense, as the two cases cannot be distinguished by the called application. There would be no advantage from just disallowing argv == NULL.

The problematic case from the security persepctive is allowing argv[0] == NULL (or more precisely argc==0). The problematic case from the compatibility perspective is disallowing argv == NULL in the execve() call. Thus only disallowing argv == NULL in the call gives us the worst from both perspectives. We get all compatibility problems with no increase in security.

Handling argc==0 in the kernel

Posted Jan 28, 2022 16:35 UTC (Fri) by larkey (guest, #104463) [Link] (25 responses)

That makes sense, I didn't know that Linux constructed the {NULL} array. However, this makes many people be confused about the original bug and the BSD behavior. E.g., FreeBSD, until 2 days ago, allowed a {NULL} array as argv but not argv==NULL.

> Yeah, but disallowing the former and allowing the latter does not make any sense, as the two cases cannot be distinguished by the called application. There would be no advantage from just disallowing argv == NULL.
>
> The problematic case from the security persepctive is allowing argv[0] == NULL (or more precisely argc==0). The problematic case from the compatibility perspective is disallowing argv == NULL in the execve() call. Thus only disallowing argv == NULL in the call gives us the worst from both perspectives. We get all compatibility problems with no increase in security.

I disagree here though. Disallowing argv == NULL but allowing argv[0] == NULL would allow for this "iterator" pattern (which is as old as UNIX):

for (**arg = &argv[1]; arg < &argv[argc]; arg++)

Handling argc==0 in the kernel

Posted Jan 28, 2022 16:57 UTC (Fri) by corbet (editor, #1) [Link] (4 responses)

Allowing argv[0]==NULL would indeed allow the "old as Unix" pattern to work. But it also works fine now in the argv==NULL case. The pattern, perhaps equally old, that fails is:

    for (arg = argv + 1; *arg; arg++)
        /* ... */

...and that pattern will still fail badly with argv[0]==NULL.

Handling argc==0 in the kernel

Posted Jan 28, 2022 17:00 UTC (Fri) by larkey (guest, #104463) [Link] (3 responses)

Arguably a bad pattern :-)

But I think that Ariadne's patch should be applied nontheless. It's just that there's much confusion about these two different issues (although related).

Handling argc==0 in the kernel

Posted Jan 29, 2022 18:56 UTC (Sat) by adobriyan (guest, #30858) [Link] (2 responses)

The best, UB free, pattern is

for (int i = 1; i < argc; ++i) { const char *arg = argv[i]; ... }

Handling argc==0 in the kernel

Posted Jan 30, 2022 23:32 UTC (Sun) by rschroev (subscriber, #4164) [Link] (1 responses)

That's simple, readable, clearly states the intent. Why do people even try to come up with other ways to do things?

Handling argc==0 in the kernel

Posted Jan 31, 2022 1:01 UTC (Mon) by jreiser (subscriber, #11027) [Link]

>Why do people even try to come up with other ways to do things?

Because that does not accommodate state changes: deleting, re-arranging, or re-writing pieces of argv (and thus argc); because the GNU getopt routine has not always existed, has not always offered all its current features, and early suffered from bugs; and probably other reasons, too.

Handling argc==0 in the kernel

Posted Jan 28, 2022 17:12 UTC (Fri) by matthias (subscriber, #94967) [Link]

> I disagree here though. Disallowing argv == NULL but allowing argv[0] == NULL would allow for this "iterator" pattern (which is as old as UNIX):
> for (**arg = &argv[1]; arg < &argv[argc]; arg++)

Disallowing argv[0] still allows for this pattern. The security problem arises with applications relying on argc>0 (like pkexec did).

And there is really no reason to disallow argv == NULL (in the execve()-call) while allowing argv[0] == NULL. As Linux constructs the { NULL } array, both things cannot be distinguished from the called application. There is no reason to get all the compatibility problems arising from disallowing argv == NULL without gaining any security benefit. Either we want the get rid of the security issues. Then we have to disallow argv[0] == NULL. Or we can leave everything as it is now.

Handling argc==0 in the kernel

Posted Jan 29, 2022 8:31 UTC (Sat) by matthias (subscriber, #94967) [Link] (18 responses)

> for (**arg = &argv[1]; arg < &argv[argc]; arg++)

I just was wondering if this is UB if argc==0. Probably it is. According to the C standard argv is a argc+1 sized array, where argv[argc] == NULL. If argc is 0, argv[1] uses an index outside of the bounds of the array. Thus even writing argv[1] would be UB. The code argv+1 instead of &argv[1] would be valid, as it is perfectly allowed to construct a pointer pointing to the address immediately after the end of an array. However one is not allowed to dereference it.

I do not really expect this to break even in the case of argc==0 and of course as long as the program is called with argc>0 this code is not UB at all. But a conforming compiler can probably eliminate a check argc!=0 that occurs later on, because the very fact that argv[1] has been used tells the compiler that either argc>0 or the code is UB and the compiler is free to do whatever it wants.

Handling argc==0 in the kernel

Posted Jan 29, 2022 10:39 UTC (Sat) by larkey (guest, #104463) [Link] (13 responses)

> However one is not allowed to dereference it.

But there's no dereferencing? But more plainly perhaps:

for (char **arg = argv + 1; arg < argv+argv; arg++)

Handling argc==0 in the kernel

Posted Jan 29, 2022 11:05 UTC (Sat) by matthias (subscriber, #94967) [Link] (12 responses)

argv[1] is clearly dereferencing and the & operator then takes the address of the dereferenced value. If you have an invalid pointer, you are not allowed to dereference it. Writing &*a is UB if a is an invalid pointer, even if the combination of & and * should be a no-op. And writing &argv[1] is UB if argv is an array of length 1 (only containing the terminating NULL) as writing argv[1] is already UB. You cannot make the UB of argv[1] be defined again by later taking the address.

I agree that the compiler will probably generate the exact same code for **arg = &argv[1] and **arg = argv + 1. But in the first case, the compiler is allowed to conclude that the length of the argv array is at least two (including the final NULL value), as argv[1] is UB if the array has length one. I am not saying that any compiler actually does this, but when it comes to UB, compilers are known to do very crazy things, so it is not entirely impossible. And in the case of the main function it is even less likely that the compiler will do anything crazy. The pointer is provided by the OS and the compiler does not know the length of the array (unless it uses the special connection between argv and argc). So the code should probably be fine. But this is one of the things I really do not like about C. The threat of UB is almost everywhere. And at some point in the future some clever compiler will exploit this against you.

Handling argc==0 in the kernel

Posted Jan 29, 2022 11:22 UTC (Sat) by mchapman (subscriber, #66589) [Link]

> Writing &*a is UB if a is an invalid pointer, even if the combination of & and * should be a no-op.

No, this is not correct.

C specifies that &*a is equivalent to a: "neither [the * operator] nor the & operator is evaluated and the result is as if both were omitted". This expression is valid for any value of a, even a null pointer.

Handling argc==0 in the kernel

Posted Jan 29, 2022 23:08 UTC (Sat) by areilly (guest, #87829) [Link] (10 responses)

"And writing &argv[1] is UB if argv is an array of length 1 (only containing the terminating NULL) as writing argv[1] is already UB."

No: &argv[1] is semantically identical to argv + 1. It is pointer arithmetic and does not cause a dereference.

However: in deference to C support on AS400 (and Unisys? anything with fancy pointer representations) it *is* undefined behaviour for arrays of length zero. It's only legal to construct a pointer value to the single element past the end of the array, whether or not the pointer is subsequently dereferenced. This is also why it's technically illegal to use a post-decremented pointer traversal that accesses the first element of an array. The one-past-the-end rule doesn't extend to one-before-the-beginning. Sigh.

Handling argc==0 in the kernel

Posted Jan 30, 2022 10:43 UTC (Sun) by larkey (guest, #104463) [Link] (9 responses)

> However: in deference to C support on AS400 (and Unisys? anything with fancy pointer representations) it *is* undefined behaviour for arrays of length zero.

Aren't zero-length arrays a GNU extension? Or do you refer to flexible struct members?

Handling argc==0 in the kernel

Posted Jan 30, 2022 11:23 UTC (Sun) by areilly (guest, #87829) [Link] (1 responses)

To be honest, I was thinking about the argc==0 example here, where the argument vector allocation was real, because it was just the first part before an explicit null and the environment variable elements. In that case I don't think that the one-past-the-end situation arises, because the memory object is the whole thing and clearly not zero length. OK, I don't think that the zero-length issue is a thing, sorry.

Handling argc==0 in the kernel

Posted Jan 31, 2022 9:50 UTC (Mon) by larkey (guest, #104463) [Link]

Ah, I see, I was just curious if I missed something :) No problem!

Handling argc==0 in the kernel

Posted Jan 30, 2022 20:21 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (6 responses)

It is legal to call malloc(0), and it is legal for malloc(0) to return a value other than NULL. If it does so, then the only thing you're allowed to do with that value is pass it to free(). In particular, you cannot add one to it, because it's functionally equivalent to a zero-length array (but I don't know if the standard actually uses the exact phrase "zero-length array").

In practice, most implementations either return NULL or convert it into a call to malloc(1). But even then, that's a one-byte array, not a one-object array, so you still can't legally add one to it unless the pointer's type has sizeof() == 1.

Handling argc==0 in the kernel

Posted Jan 30, 2022 20:23 UTC (Sun) by NYKevin (subscriber, #129325) [Link]

> then the only thing you're allowed to do with that value is pass it to free().

Or realloc(), I suppose. Point is, you definitely can't dereference it.

malloc(0)

Posted Jan 31, 2022 1:20 UTC (Mon) by jreiser (subscriber, #11027) [Link] (3 responses)

>It is legal to call malloc(0), and it is legal for malloc(0) to return a value other than NULL. If it does so, then the only thing you're allowed to do with that value is pass it to free()

One of the uses of malloc(0) is to serve as an analogue of the Lisp function (gensym): return a unique value, one that does not point to anything else returned by malloc, both now and for the remaining life of the process. (Yes, the standard requires this.) An implementation for which malloc(0) always returns NULL is not acceptable. So in practice, many implementations begin with something like size += (0==size), i.e. malloc(0) is treated as malloc(1).

malloc(0)

Posted Jan 31, 2022 1:31 UTC (Mon) by ABCD (subscriber, #53650) [Link]

An implementation for which malloc(0) always returns NULL is expressly permitted by the POSIX standard:

If the size of the space requested is 0, the behavior is implementation-defined: the value returned shall be either a null pointer or a unique pointer.

The C17 standard says something similar:

If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned to indicate an error, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object.

malloc(0)

Posted Jan 31, 2022 9:43 UTC (Mon) by NYKevin (subscriber, #129325) [Link] (1 responses)

The null pointer satisfies all of the technical requirements of a heap-allocated zero-length array (you can form a pointer that is one past the end, you can iterate over it zero times with a standard for loop, and you can pass it to realloc without breaking anything), and all heap-allocated zero-length arrays are functionally identical because they have no state which can be mutated, so returning NULL can be thought of as semantically equivalent to a copy elision (i.e. you could think of all heap-allocated zero-length arrays as "copies" of the zero-length array that lives at NULL, and then you can elide those copies because zero-length arrays are immutable, so making a copy is unnecessary).

The fact that it happens to vaguely resemble some feature of Lisp, but the standard does not actually require it to fulfill all the requirements of that feature is, frankly, Lisp's problem, not C's problem.

malloc(0)

Posted May 28, 2026 11:13 UTC (Thu) by alx.manpages (subscriber, #145117) [Link]

> The null pointer satisfies all of the technical requirements of a heap-allocated zero-length array

This is not correct. You're allowed to do pointer arithmetic on malloc(0) if and only if it returns a non-null pointer. p+0 is valid for any non-NULL p, but NULL+0 has undefined behavior. This limitation of NULL pointers was lifted recently (ISO C2y), but historically it was UB; also, that change might have unexpected consequences, such as lower quality of static analysis, so I suggest not relying too much on it.

P.S.: Sorry for resurrecting this. I was researching the guarantees on argv[0] and read this incorrect statement by chance.

Handling argc==0 in the kernel

Posted Jan 31, 2022 10:02 UTC (Mon) by larkey (guest, #104463) [Link]

Fair enough, although I *think* C restricts the meaning of array to things that are declared T D[N]; Zero-allocations are definitely possible though, yes, and POSIX is a bit more lax about wording since they say "argv is an array" while, in C terminology, it's "just" a pointer to some memory that... and so on.

Anyway, the POSIX standard pretty clearly says that argv is NULL-terminated, that is, not by itself a zero-length array, but at least

char **argv = { NULL };

Handling argc==0 in the kernel

Posted Jan 30, 2022 3:24 UTC (Sun) by nybble41 (subscriber, #55106) [Link]

> If argc is 0, argv[1] uses an index outside of the bounds of the array. Thus even writing argv[1] would be UB.

Another comment by areilly hinted at this already, but evaluating argv[1] for an argv array of size 1 (where argc == 0) is not undefined behavior. You are allowed to construct (but not dereference) a pointer to the element immediately after the end of an array. (Taking the address of a dereference or array indexing expression does not count as actually dereferencing the pointer: even though argv[1] in an expression by itself would be UB, &argv[1] and &*(argv + 1) are both semantically identical to the pointer arithmetic operation argv + 1. The address-of operator "cancels out" the dereference.) So there is no undefined behavior in the example, even if argc == 0, as &argv[1] and &argv[0] are both valid pointers.

Handling argc==0 in the kernel

Posted Mar 28, 2022 21:41 UTC (Mon) by fest3er (guest, #60379) [Link] (1 responses)

"According to the C standard argv is a argc+1 sized array, where argv[argc] == NULL. If argc is 0, argv[1] uses an index outside of the bounds of the array."

Hmmm. Off-by-one seems to rear its head here. If argc is 0, then argv[] should contain 0+1 elements, thus only argv[0] exists and should be null and argv[1] is out-of-bounds.

Aside, why are people saying that argv is the same as argv[0]? Isn't argv the address of the array, and argv[0] the address of the zeroeth argument (or null to indicate the end of the array)? Thus, shouldn't argv always be a valid pointer to an array and the array always contain at least the terminating NULL pointer?

And when the kernel constructs the argv array, does it use its memory or the user's memory?

Handling argc==0 in the kernel

Posted Mar 29, 2022 12:51 UTC (Tue) by jem (subscriber, #24231) [Link]

>Hmmm. Off-by-one seems to rear its head here. If argc is 0, then argv[] should contain 0+1 elements, thus only argv[0] exists and should be null and argv[1] is out-of-bounds.

Yes, of course. If argc is 0, then the argument vector is empty and only contains the terminating NULL at index 0. You can't expect to find anything of interest in argv[1].

>And when the kernel constructs the argv array, does it use its memory or the user's memory?

The argv array is in the process (user) address space. You can print the address of argv if you are interested to find out where it is located.

Handling argc==0 in the kernel

Posted Mar 29, 2022 20:16 UTC (Tue) by nybble41 (subscriber, #55106) [Link]

> The code argv+1 instead of &argv[1] would be valid, as it is perfectly allowed to construct a pointer pointing to the address immediately after the end of an array. However one is not allowed to dereference it.

From C99 (draft) §6.5.3.2:

> If the operand is the result of a unary * operator, neither that operator nor the & operator is evaluated and the result is as if both were omitted, except that the constraints on the operators still apply and the result is not an lvalue. Similarly, if the operand is the result of a [] operator, neither the & operator nor the unary * that is implied by the [] is evaluated and the result is as if the & operator were removed and the [] operator were changed to a + operator.

So "&argv[1]" is effectively rewritten as "argv+1" and there is no undefined behavior either way. You could even have an expression like "&(*p)" or "&p[0]" where p == NULL and the result will be well-defined (NULL). A conforming compiler cannot assume that argc > 0 based on the presence of "&argv[1]".

Handling argc==0 in the kernel

Posted Jan 28, 2022 20:07 UTC (Fri) by khim (subscriber, #9252) [Link] (1 responses)

You found the right page, just, perhaps, not the right quote. I think that one should close that discussion about meanings of different words:

> The wording, in particular the use of the word should, requires a Strictly Conforming POSIX Application to pass at least one argument to the exec function, thus guaranteeing that argc be one or greater when invoked by such an application.

I don't think explanations can be more clear than that.

You don't need to dig around for strict meanings of different words, authors of the standard did that for you.

Handling argc==0 in the kernel

Posted Jan 28, 2022 21:13 UTC (Fri) by larkey (guest, #104463) [Link]

But in what way does this contradict what I say? "should" means that this is a recommendation, i.e., a *strictly* conforming application needs to fulfill this criterion. But POSIX only uses should for argv[0] being the program name, but they use *shall* for argv ≠ NULL. Which is precisely my point. The latter is a *must do*, the former is not as strict as a requirement.