| View previous topic :: View next topic |
| Author |
Message |
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
 Posted: Tue Feb 07, 2017 8:06 pm Post subject: |
 |
|
i would disallow myself b and d format.
if you use ":" only "cmd:_" or "cmd:$" are valid (no space between command and ":"), it will even ease parser as you need to look at next char after a command, is it "_" or ":" or "$"
i don't know if "cmd:_args" or "cmd:args" should be both valid. I suppose it could if you still parse only one char after command, you can assume "start: --args" as cmd:<start> and args:<_--args>, i don't think any program will dislike a param starting with a space. |
|
| Back to top |
|
 |
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Tue Feb 07, 2017 8:24 pm Post subject: |
|
|
The shell argc / argv construction mechanism does most of the parsing for us, throwing out leading and trailing whitespace. The only question (with the "krinn approach", anyway) is whether we recognize ':' as a separate argv value or only when appended to a valid command.
The construct "cmd:arg" as laid out by Mr. krinn himself, looks like an argument, or else an undefined command, not a command with an argument. 
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Tue Feb 07, 2017 8:54 pm Post subject: |
|
|
Yeah, you start to get into lexing vs standard option parsing.
Really the pair of you are confirming my feeling against colons (which are not that rare ime) as a special introducer.
Any halfway experienced C coder is used to handling '--', as it's required by POSIX (which doesn't mandate longopts.) |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Tue Feb 07, 2017 9:25 pm Post subject: |
|
|
Part of my objection to it is that your proposed use of '--' doesn't always feel like a "special introducer" to me, a term which I take to mean something along the lines of, "Things after this point are different." Am I close?
Its first occurrence is used to signal that commands take arguments. Wouldn't it be more natural, more consistent with *nix conventions, if this is the right approach, to use a regular option, for instance -a and --args, to indicate that commands on this invocation take arguments? Its second occurrence is used to signal the end of the current argument list—and possible start of a new command and optional associated arguments, so it's being used as a separator here, not as a special introducer, if I understand correctly, and '--' is never used as a mere separator with any other *nix tool that I know of. Anyway, that's what makes it look odd to me.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Last edited by John R. Graham on Wed Feb 08, 2017 12:46 am; edited 3 times in total |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Tue Feb 07, 2017 11:02 pm Post subject: |
|
|
Looking back it appears I missed an earlier post, my bad. | steveL wrote: | | Getting there; I like \; from find usage, but as stated I don't think it's a good idea here as eval is used currently at a lower layer (I've always wanted to get rid of its current incarnation), and the potential for a fumbled ..param\; is too great. |
| John R. Graham wrote: | | I dug into this assertion and I think I've discovered that it shouldn't matter. One of the elegant things about this parsing we're talking about is that none of these extra symbols make it to the init scripts: it's all filtered out in the command line parser in "openrc-run.c". I think we can do what we want, syntax-wise, without running the risk of breaking things lower down. |
What assertion? There is a greater risk of mistaken semicolons in the input, with that option.
Forgive me, but I've heard words along the lines of "I think I've discovered that it shouldn't matter," far too often in #bash and on the forums, usually when someone is trying to justify not quoting parameter expansions in a very specific case, instead of scripting robustly once.
semicolons and eval never mix. While we cannot disallow semicolons in general, we can sure as hell avoid forcing the user to put them in, at a time when they are doing things manually by definition (so errors are more likely than not) and we are passing the parameters to eval (gack.)
"Parsing" doesn't come into it; command-line parameters can be anything, or we end up with blunt tools. (I feel bad enough not allowing a scripter to avoid possible '-*' paths interfering with lower-level command options; but this is for *nix admins at the console.)
| John R. Graham wrote: | | Part of my objection to it is that your proposed use of '--' doesn't always feel like a "special introducer" to me, a term which I take to mean something along the lines of, "Things after this point are different." Am I close? |
Yup.
| Quote: | | Its first occurrence is used to signal that commands take arguments. Wouldn't it be more natural, more consistent with *nix conventions, if this is the right approach, to use a regular option, for instance -a and --args, to indicate that commands on this invocation take arguments? Its second occurrence is used to signal the end of the current argument list—and possible start of a new command and optional associated arguments, so it's being used as a separator here, not as a special introducer, if I understand correctly, and '--' is never used as a mere separator with any other *nix tool that I know of. |
It's a "one-off" separator, ie introducer of args, for most POSIX utils; not a "field-separator" as I read "mere separator" above.
I gave the example earlier: | Code: | | mv -f -- "$old" "$new" |
I just can't find the POSIX command-line utility guidelines right now (as I still need to reinstall.) Certain utilities don't conform, like find, due to historical practice (or purpose), but that has to be called out in the description.
| Quote: | | Anyway these are the reasons that the syntax seems odd to me. |
Feeling is an odd one, and we could brainstorm all kinds of "syntax" for what is meant to be kept simple. I favour '--' simply because it is a standard thing.
Secondly, it is so simple to code for. (And C coders are already used to looking for it, certainly when they process options themselves, which does happen.)
So from (standard) one-off introducer, extended to first separator, as it's already used somewhat like that, and it's the kind of thing we can do quickly on the command-line (so fingers are now in gear for separating the next command); an option works as well, though, so I'd be happiest with both (since we still need a separator.)
Seems like we're getting closer though; an option in both cases, and either: a separator (that can take the place of the option) or: some sort of more complex logic.
What it comes down to: I'm fed up of utilities inventing "syntax" (confusing lexing with parsing, as ever) and thought we'd got past it decades ago. I've done my time learning esoteric uses of find (and more importantly, how to script robustly against it.) I don't see the point (nor justification) in adding anything tricky to openrc; it would just be repeating the same mistakes everyone else has done so many times, for what is meant to be a rare use-case.
Keep It Simple and Stupid. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Tue Feb 07, 2017 11:15 pm Post subject: |
|
|
| steveL wrote: | | John R. Graham wrote: | | I dug into this assertion and I think I've discovered that it shouldn't matter. One of the elegant things about this parsing we're talking about is that none of these extra symbols make it to the init scripts: it's all filtered out in the command line parser in "openrc-run.c". I think we can do what we want, syntax-wise, without running the risk of breaking things lower down. |
What assertion? |
Your assertion that the semicolon, if included in the proposed feature's syntax, would ever reach the evals in the init scripts under any circumstances, ever. Now if you think semicolon is dangerous in general because it has to be escaped to lose its special meaning, I can relate to that. But, in my defense, you said you liked it.
Also, as you were composing your reply, I was refining my post above. I think it's only fair that I edit it back to exactly what you were replying to, and I'll do that shortly. Edit: Done.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Last edited by John R. Graham on Tue Feb 07, 2017 11:23 pm; edited 1 time in total |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Tue Feb 07, 2017 11:23 pm Post subject: |
|
|
| steveL wrote: | | Code: | | mv -f -- "$old" "$new" |
I just can't find the POSIX command-line utility guidelines right now (as I still need to reinstall.) Certain utilities don't conform, like find, due to historical practice (or purpose), but that has to be called out in the description. |
That's my point against -- because it's a legit usage for args on tools and you will have hard time to diff if -- was for the initscript or args for it.
Look at your mv example, how would the parser unscramble this?
| Code: | | start -- -f -- "$old" "$new" |
when this make it easy | Code: | | start: -f -- "$old" "$new" |
|
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Wed Feb 08, 2017 1:13 am Post subject: |
|
|
| steveL wrote: | | I just can't find the POSIX command-line utility guidelines right now (as I still need to reinstall.) Certain utilities don't conform, like find, due to historical practice (or purpose), but that has to be called out in the description. |
I think this is probably what you're looking for: | Quote: | Guideline 10:
The first -- argument that is not an option-argument should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the '-' character. |
Reading this, it's not completely obviously to me that you're proposing a compliant usage. The intent seems pretty clear: everything after the -- should be the same type of thing (operand), including following instances of --.
Just as an exercise, if we're trying to compose a version of rc-service that was completely compliant with the POSIX Utility Syntax Guidelines, didn't have to worry about backwards compatibility, and supported multiple service commands with options for each command, we'd end up with something like this, I think: | Code: | | rc-service mysql -c stop --foo bar -c start --bar baz |
where -c indicates which arguments are service commands. Would this in your mind be POSIX compliant or, perhaps more importantly, eval friendly?
In fairness, I think the current command line syntax of rc-service is POSIX Guideline compliant. Don't you? It's shoe-horning the argument support in while trying to maintain backwards compatibility that's causing the tension.
Also, I'm curious. Was I confusing lexing and parsing, in your opinion?
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed Feb 08, 2017 8:56 am Post subject: |
|
|
I have rethink about it, the trouble is keeping "multi-command" in one line and passing args to them.
While openrc accept multi-command, it would just make everything easy to accept only one command with args.
Two running mode, multi-command no args or one command with args ; you remain compatible, you can use whatever you like as separator, you only loose ability to run multi-command in one line if your command have args.
And the parsing will be a piece of cake: have -- ? take what is prior -- as command, take what is after -- as args ; if no -- it's multi-command parsing, just as of today. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Wed Feb 08, 2017 1:22 pm Post subject: |
|
|
If I'm interpreting the syntax guidelines properly, at least it's compliant usage, but don't give up just yet. It seems to me thatandandall have a lot in common and are approximately equivalent in complexity of implementation. There's a simple path to backwards compatibility and multi-command support with the first two; there's POSIX guidelines compliance with the last two. Calculation of the intersection of these two sets is left as an exercise for the reader. I happen to like your original syntax proposal, as I think it's easy on the eyes, but Steve has a point.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Feb 09, 2017 12:04 am Post subject: |
|
|
| steveL wrote: | | What assertion? |
| John R. Graham wrote: | | Your assertion that the semicolon, if included in the proposed feature's syntax, would ever reach the evals in the init scripts under any circumstances, ever. Now if you think semicolon is dangerous in general because it has to be escaped to lose its special meaning, I can relate to that. But, in my defense, you said you liked it. ;-) |
Ah, I see the confusion; I only "liked" it, and saw where Hu was coming from, in relation to its extant usage in find, which uses it to terminate an -exec(dir) though + is usually better (and standard in POSIX since at least 2001, though I don't have my SUS2 docs to hand.)
Nonetheless, my point stands: if we force the user to use the semicolon as a delimeter, a) they have to escape it in the shell (same as find). so it's less convenient, and b) at some point they're going to miss a space out, or even two, because it's a human, and as humans we know we're more likely to fumble when we're stressed, or doing something unusual, which this is by definition. At that point, the semicolon is no longer a separate parameter, but it is intermixed with "extra_command_opts" which later get passed to eval.[1]
So, this option is getting thorny already, and I don't see the point in needing to sanitise inputs; speaking for the CPU, it's a waste of cycles we've only brought on ourselves (or would have, if we used the semicolon.)
I think the key thing to bear in mind, is that we're starting from a simple base of "cmd [cmd].." where each cmd token is a standard identifier (in lex terms) matching one of the defined commands in the script (predefined defaults + extra_commands + extra_started_commands..)
Arguments have never been supported before, and the environment is locked-down, to keep things "fire-and-forget" and secure.
So by definition, we're only adding arguments for the administrator at the console, which get added to an existing setup, configured command call; not for scripting purposes with any random input filenames on the fs.
OFC the road to hell is paved with good intentions.. while your average bashrc alias newcluster='/etc/init.d/mysql -- restart --wsnewcluster' is not an issue, once you open the door, someone is bound to cut their fingers on a sharp tool.
*shrug* they're *nix admins ffs.
--
[1] in an implementation I dislike because it is too lax, in order to allow evaluation of fd redirection. iirc this is only used by udev for debug mode; but start-stop-daemon has command-line parms for standard fd redirection, so the laxity is not even needed for its one usage in official intiscripts.
edit: fix quote tag, s/input/fd/
Last edited by steveL on Thu Feb 09, 2017 1:25 am; edited 2 times in total |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Feb 09, 2017 12:31 am Post subject: |
|
|
| Code: | | mv -f -- "$old" "$new" |
| krinn wrote: | That's my point against -- because it's a legit usage for args on tools and you will have hard time to diff if -- was for the initscript or args for it.
Look at your mv example, how would the parser unscramble this?
| Code: | | start -- -f -- "$old" "$new" |
|
There is no "parser"; there's a tiny loop, usually in main(), that processes command arguments, usually after a looped call to getopts(), which leaves us with non-option arguments.
We break on "--" (consuming it) since that makes sense: for most utilities, those arguments are filenames. A scripted call as shown above, needs to separate options from unknown input filenames, so those arguments aren't treated as options, but fail as filenames,
(No argument: usually means read stdin.)
Our arguments are command tokens, and we're extending it to add parameters at the command line, for the admin to add basic options (though sure, they're going to do what they want.)
Do we really need to support variable random filepaths from untrusted inputs, when the conf.d is already setting those up for us, or the admin is going to add them with tab-completion?
What daemon takes a list of files to process, like a utility, requiring this, rather than the usage of "-x filepath" for a conf or whatever?
If so, I'd prefer --! to delimit the next command (as stated before.)
Also, the '--' comes before the cmd, not after.
| Quote: | when this make it easy | Code: | | start: -f -- "$old" "$new" |
|
How do we know when the next command token starts? (so we can check it against the script-defined commands.) I'm restricting "--" which has no place in the existing setup wrt backward compatibility (all commands are simple identifiers); or using "--!" if we must be robust for scripted purposes (since exec-chaining is the place of the initscripts, it has no place here.)
You're restricting an extensible set of identifiers with following ':' -- how do you know none of them will ever turn up in usage for some service down the line?
More importantly how do you guard against random inputs, as you want me to do? |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Thu Feb 09, 2017 1:35 am Post subject: |
|
|
| steveL wrote: | | ...How do we know when the next command token starts? (so we can check it against the script-defined commands.) |
I've had this in the back of my mind as a possible implementation since the beginning but I was tacitly avoiding it because I thought it was a little too cute. If you have to check the tokens against the list of valid commands, then you don't need a separator at all. The complete list of valid commands is discoverable, as I think you realize, for each init script.
| steveL wrote: | | There is no "parser"; there's a tiny loop, usually in main(), that processes command arguments, usually after a looped call to getopts(), which leaves us with non-option arguments. |
You know that getopts() is the parser, right?
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Feb 09, 2017 1:44 am Post subject: |
|
|
| John R. Graham wrote: | | I think this is probably what you're looking for |
Ah yeah, thanks.
| Quote: | Guideline 10:
The first -- argument that is not an option-argument should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the '-' character. |
| Quote: | Reading this, it's not completely obviously to me that you're proposing a compliant usage. The intent seems pretty clear: everything after the -- should be the same type of thing (operand), including following instances of --.
Just as an exercise, if we're trying to compose a version of rc-service that was completely compliant with the POSIX Utility Syntax Guidelines, didn't have to worry about backwards compatibility, and supported multiple service commands with options for each command, we'd end up with something like this, I think: | Code: | | rc-service mysql -c stop --foo bar -c start --bar baz |
where -c indicates which arguments are service commands. Would this in your mind be POSIX compliant or, perhaps more importantly, eval friendly? |
It is those, but what happens when we want to pass the -c option to our daemon process?
To match sh, where I think you're coming from, it'd be: -c stop '--foo bar' -c start '-c --bar baz' or: -c start -c on its own would still work (assuming required parm.)
Either way, it's still thorny; the essential problem is a delimeter that we won't ever use; and daemons aren't command-line utilities, by definition.
| Quote: | In fairness, I think the current command line syntax of rc-service is POSIX Guideline compliant. Don't you? It's shoe-horning the argument support in while trying to maintain backwards compatibility that's causing the tension.
Also, I'm curious. Was I confusing lexing and parsing, in your opinion? |
The discussion was, as so often happens when people start to talk about "parsing" command-line arguments or options; but then I fell into the same pitfall myself, with "standard option parsing."
s/parsing/processing/ and we all make a lot more sense, so let's just pretend we all used that term instead.. ;-)
| John R. Graham wrote: | | If I'm interpreting the syntax guidelines properly, at least it's compliant usage, but don't give up just yet. It seems to me thatandandall have a lot in common and are approximately equivalent in complexity of implementation. |
Last one should read:
As stated above, the -c option doesn't work in general, as it means we can't use a -c command line option to a daemon very easily.
Last one is much easier to implement imo. At minimum, you have to find and NUL the colon, to set the string to compare when using colons, and you have more complex logic too. '--' is just a simple "gather til delimiter or end", and we just NULL the "--" out of the way to make an an argv vector.
| Quote: | | There's a simple path to backwards compatibility and multi-command support with the first two; there's POSIX guidelines compliance with the last two. |
We're not pretending to be a standard utility, nor are we in any way concerned with filenames as operands, only as optional configuration parameters.
Backwards compatiblity is same for all, as currently no arguments are supported at all, only simple command identifiers.
| Quote: | | I happen to like your original syntax proposal, as I think it's easy on the eyes, but Steve has a point. |
Keeping it even simpler, why don't we just forget about multi-command when using -a (or whatever) for args?
No delimiter required, and we can use -- to our daemon if we must.
After all, it's for exceptional situations, not everyday usage; and a script can always use multiple calls (in fact it'd likely be easier and more robust, to do so.)
So | Code: | | /etc/init.d/mysql -a start --wsnewcluster |
is your lot, and off we go (very easy to do.) |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Feb 09, 2017 2:05 am Post subject: |
|
|
| John R. Graham wrote: | | If you have to check the tokens against the list of valid commands, then you don't need a separator at all. |
Yes, you do (which is why Hu brought one up): command arguments can be anything. We want a minimally restrictive delimiter (if we must support multiple commands); not an extensible set that could conflict with feasible parms down the road.
| steveL wrote: | | There is no "parser"; there's a tiny loop, usually in main(), that processes command arguments, usually after a looped call to getopts(), which leaves us with non-option arguments. |
| Quote: | | You know that getopts() is the parser, right? |
We're cross-posting (it's an option-processor.. ;)
Write one (or process options manually) and you'll see: it's only minimally like lexing, which comes before parsing. I'd call it "string processing" at most.
I recommend the K&R exercises; if you've not done 1-23, you should. It flows nicely into the next one; so begins basic lexing (even if we just skip everything, and don't really process most tokens.) |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Feb 09, 2017 2:14 am Post subject: |
|
|
| krinn wrote: | | I have rethink about it, the trouble is keeping "multi-command" in one line and passing args to them. |
Ah missed this; yeah you got it.
| Quote: | While openrc accept multi-command, it would just make everything easy to accept only one command with args.
Two running mode, multi-command no args or one command with args ; you remain compatible, you can use whatever you like as separator, you only loose ability to run multi-command in one line if your command have args. |
Yeah.
| Quote: | | And the parsing will be a piece of cake: have -- ? take what is prior -- as command, take what is after -- as args ; if no -- it's multi-command parsing, just as of today. |
Well, it goes before (t's always easier to do something after a recognised token, ime); but if we have distinct modes, we just need the option. (and ofc the associated sh bits.) |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Thu Feb 09, 2017 2:19 am Post subject: |
|
|
I think we're back in order now. | John R. Graham wrote: | | If you have to check the tokens against the list of valid commands, then you don't need a separator at all. |
| steveL wrote: | | Yes, you do (which is why Hu brought one up): command arguments can be anything. We want a minimally restrictive delimiter (if we must support multiple commands); not an extensible set that could conflict with feasible parms down the road. |
This is a circular argument, I think, because -- is also a potential argument down the road. If you want to be able to pass all possible arguments without restriction, and I think you do, then you need a (seldom used) escaping mechanism in either case.
Edit: Acknowledged that -c isn't a good choice. You seem to be arguing above, though, that strict POSIX guidelines compliance isn't strictly required, even though you spent a fair amount of time earlier arguing that it was, which kind of leads me back to colon as a superior delimiter from the perspective of semantics. It's not the simplest implementation, but it'll be pretty simple.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Last edited by John R. Graham on Thu Feb 09, 2017 3:34 am; edited 1 time in total |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21633
|
| Posted: Thu Feb 09, 2017 2:30 am Post subject: |
|
|
| As a quick aside, when I suggested semicolon, I had not looked at (and still have not looked at) the internals you two are discussing patching. I picked it because, as steveL recapped, any delimiter that gets dropped into the middle of an existing language needs to be very unlikely to be otherwise valid in the language. I did not know at the time that the data is processed in a way that makes some characters more complicated than others. In light of that quirk, semicolon may not be a good choice. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Thu Feb 09, 2017 9:42 am Post subject: |
|
|
| John R. Graham wrote: | | Code: | | MY_ARGS="--wsrep-new-cluster" rc-service mysql start |
Does this work for you? It doesn't work for me with openrc-0.22.4. In fact, nothing from the shell environment seems to leak through to the init script. It would seem reasonable to me that it would work, considering how other Gentoo tools handle the environment, but it doesn't. |
I didn't have time to read the whole thread, so I am sorry if this already has been answered.
The reason why this is not working is a security feature: openrc explicitly filters all shell variables except for those explicitly allowed in /etc/rc.conf.
In /etc/rc.conf there is a possibility to allow certain variables explicitly, but I would have to look up the details. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Thu Feb 09, 2017 2:34 pm Post subject: |
|
|
Yeah, sorry. We're discussing the relative trivialities of syntax and it's gotten long. From earlier in the thread: | John R. Graham wrote: | | Since I didn't know either way, I took the trouble to verify what was going on. What's actually happening is that init scripts are run from a fresh /etc/profile environment; so you can push something into the init script environment by making an entry in /etc/env.d/, but not from the running shell environment. |
It does filter out everything that's not on the whitelist (which, by default, is almost empty: there's a small internal whitelist to which the rc_env_allow variable in /etc/rc.conf is additive), but then it adds the whole /etc/profile environment back. On my first pass through the code, I didn't identify the rc_env_allow mechanism, though, so thanks!
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Mon Feb 27, 2017 12:05 am Post subject: |
|
|
I have completed a proof of concept implementation of the modified syntax. As soon as Infra gets my overlay set up I'll make this available via Layman, but for the time being if you want to take a look, the work is available on a fork of openrc on GitHub. I'm working on three syntax variations:- The krinn syntax is on branch script-cli-args.
- The steveL syntax is on branch double-dash.
- The single command per invocation syntax, introduced by -a, is on branch dash-a.
All three implementations preserve backwards compatibility in all respects. Interestingly, the patch files for the two "delimiter" approaches against current stable openrc differ in length by only 12 bytes. #3 is a bit longer because more changes have to be made to pass the -a option from rc-service to openrc-run.
Obviously there's a major increase in complexity involved in parsing the colon!  Pulling my tongue resolutely out of my cheek, all three are minor changes with the major effort expended becoming familiar with the code base.
They're all three basically feature complete, man pages and everything, but I'm still working on a few additional enhancements:- Provide a mechanism for init scripts to advertise that they can use arguments, erroring out (or at least warning) on those that can't if arguments are present. Just so you know, arguments are completely harmless to init scripts that don't use them, but it seems the right thing to do. (Such a mechanism should require no modification to init scripts that do not take arguments.)
- Enhance the built-in usage description, which is somewhat incomplete.
- For completeness, implement an escaping mechanism for the unlikely event that required arguments collide with the new syntax. Not required for #3 above, which is a point in its favor.
Comments, scathing critiques, accolades, or anything in between all welcome.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Wed Mar 08, 2017 6:58 pm Post subject: |
|
|
In the furtherance of (a) and (b) above, I'm expanding an existing mechanism that openrc-run uses to collect information on the init script. The init script is already sourced and the description variable is echoed and then collected. The following is a generic expansion of that (very) short shell script so that it can be easily expanded to collect arbitrary information: | rc-service-info.sh: | #!/bin/sh
#
# Put variable names here that openrc-run should collect:
#
SVC_INF_VARS="description allow_args"
#
# These three variable names define service script specific
# commands. These are special because a command description
# variable may be defined (and should be collected).
#
SVC_CMD_VARS="extra_commands extra_started_commands extra_stopped_commands"
conditional_emit_var() {
VARN="$1"
VARV="${!VARN}"
if [ -n "$VARV" ] ; then
echo "$VARN=\"$VARV\""
fi
}
if [ -z "$1" ] ; then
exit 0
fi
if ! rc-service -e "$1" ; then
exit 0
fi
source $(rc-service --resolve "$1")
for i in $SVC_CMD_VARS $SVC_INF_VARS; do
conditional_emit_var "$i"
done
for i in $SVC_CMD_VARS ; do
VAR="${!i}"
for j in $VAR ; do
conditional_emit_var "description_$j"
done
done |
@steveL, would you critique this for me, please? There are a couple of bashisms that I don't know how to eliminate without resorting to the reviled eval. What it does is print out name=value tuples for several listed or derived variable names within the init script. The single command line argument is the init script name, with or without path. For example: | Code: | $ sudo ./rc-service-info.sh hwclock
extra_commands="save show"
description="Sets the local clock to UTC or Local Time."
description_save="Saves the current time in the BIOS."
description_show="Displays the current time in the BIOS." |
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Tue Mar 21, 2017 11:36 pm Post subject: |
|
|
| John R. Graham wrote: | | There are a couple of bashisms that I don't know how to eliminate without resorting to the reviled eval. What it does is print out name=value tuples for several listed or derived variable names within the init script. |
Heh, this is why I like bash ;-)
bash enables the most common, sometimes legitimate, uses of eval, as here: indirect variable references. The inverse facility to expansion, assignment, is provided by the printf builtin, eg: printf -v var '%s' "$*"
We have to resort to eval in this instance, since variable identifiers are a) compound (concatenation), and b) come from the initscript.
Because they're external, we must check them for validity, as we do for all such uses in our scripts, since humans make mistakes.
I'd do something like | Code: | conditional_show_var(){
is_id "$1" || die "bad varname: '$1'"
eval "VARV=\$$1"
[ -n "$VARV" ] || retun
printf '%s="%s"\n' "$1" "$VARV"
:
}
is_id(){
case $1 in
''|[![:alpha:]]*|*[![:alnum:]_]*) false
esac
} |
You can use the return status of conditional_show_var, eg to return from the caller on an empty value.
We're using the fact that a 'case' that doesn't take any branches (and has no 'default') evaluates to true, as does an 'if' that has no 'else', where no branch is taken. As does a branch of a case with no command.
These are all very useful in makefiles. (You gain real insight into sh, by exploring how it is used in make, ime.)
We tend to ban single letter identiifiers as well, since we reserve them for locals; a variable used across functions must have more meaning, and cannot reliably be local in any case.
Add ? as a case value to implement that; ie: | Code: | | ''|?|[![:alpha:]]*|.. |
Note you need to disable history expansion set +H in order to use ! as a normal char in terminal (this doesn't work on zsh for some reason.) Only an issue in interactive shell, ofc, not in script files. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Wed Mar 22, 2017 7:00 am Post subject: |
|
|
| John R. Graham wrote: | | source $(rc-service --resolve "$1") |
A less known incompatibility: "source" is incompatible; "." is the POSIX syntax. Depending on the shell (e.g. in zsh), "source" might have additional features like searching in the current directory first; in other shells like dash "source" is not known at all.
Also it is unspecified whether the current "$@" is passed as arguments or whether the parameters after the source command are passed; so if in doubt and "$@" should be empty for the sourced code, it might make sense to call something like in advance. (In theory should do the same, but set -- with an empty argument list is buggy in some classical shells.) On the other hand, if you want to make sure that "$@" is passed, I would call | Code: | | source "$(rc-service -resolve "$1")" "$@" |
Edit: If you need "source" several times, it might make sense to put it in a function: | Code: | | my_source() { . "$@"; } |
Then my_source foo will guaranteed pass an empty parameter list while my_source foo "$@" wlil pass the current parameters. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Wed Mar 22, 2017 8:14 pm Post subject: |
|
|
Is true not portable in some cases? It is mentioned in the POSIX Shell spec.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
