how much integration of ss pilot file do you want to achieve?

[backamblock]

Good Morning :)
i tested and thought about slickstack the last weeks and since i now already use your script and it saves so much time and effort, i might as well help out with integrating the features that you plan for the future, as far as my knowledge and ability to google stuff goes... i am pretty good at googling and fixing stuff 😄
So for the bash scripts i should be able to do something useful here.

i don't know if you like my attempt of fixing the rclone integration yet, but i am looking at the pilot file right now and want to check in if my idea of your plan actually matches your plan.

So what's the plan? i would:
create a file ss-update-pilot - copy ss-worker and use it as a layout template, trying to recreate your best practises
create links to TIMESTAMP, MIRRORs, TMP, PATH, in ss-functions
remove the pilot file part from ss worker and instead let it call the ss-update-pilot
add a function to retrieve and use the pilot file as ss-config on ss-install if provided - either as var set before or inline-arg

how would the pilot file look?
it will be the exact ss-config-sample.txt, plain and simple, no extra files
all really one-machine-specific values are not changed on update of course - users, passwords,...
apart from that, everybody is free to keep the recommended default settings or change them
this approach will keep things simple in the future and users can copy and edit to their likings

why change the install file?
as i understand it, you want the pilot file to update a bunch of servers without logging in to each, also you want stability in terms of matching settings on a bunch of servers so you can easily recreate what is caused by what.
this way, you (and all future users) need to only create the ss-config once and have it applied to all servers in a matching group, while also easily install new servers with that exact config.
different server groups can be managed by different pilot files

how will that work?
first thing the installer checks right now is if a config is provided, after that there would be a check if a pilot file link is provided.
if no pilot is linked, everything continues as it does now.
if a link to a pilot file is provided, it will download the current ss-config and replace all values with the provided pilot file. this is done as a sanity check to prevent errors based on users cleaning the pilot file from things they won't change. this will also prevent errors based on updates to the ss-config where users have an old version as their pilot file.

what about users and passwords?
in my script i have all of these be generated randomly at each install with the same openssl function, but quite longer or shorter based on the use case. for example the SUDO_USER is pre-set to a fixed value by me. For security reasons, all other things are randomly generated: usernames for db, guest, sftp, etc are 10 characters, passwords are 64 characters, db prefix is 5 characters. this would be integrated into the ss-install and be combined with the values from the pilot file.
ssh keys? can be provided in the pilot file as a string value. will be echoed into authorized-keys.
the 64 character password will stay activated as a backup if a user changes ssh keys in the future (broken laptop, leaked keys, ...)

what about ss-worker?
it will call the same ss-update-pilot bash file that is called by the ss-install and replaces the values. just the random generator part is purely in the ss-install. i have not crawled through every file, but as far as i came, at least all the intervals and the contained functions would be able to update without use of ss-update. if more intervals are enabled, more things would be able to update.

why do all that?
imho this is a great addition to the easy deployment and keep it simple to manage principle of slickstack.
right now, slickstack manages a big big part of the install and management. the pilot file will be the missing link between having a fresh vps, creating the config and being able to easily manage even 2,3 or 100. it will fill out the gaps for more tech-savvy users.

i will stop here, the rest is more on the automatic install side and exceeds the pilot file part or what i understand as the idea behind it.

please tell me what you think, or tell me to stop bothering you with all these commits, either way is fine 👍🏻
have a nice day

[jessuppi]

Thanks for the detailed feedback and ideas @backamblock this is really what we've needed, more discussion and opinions from the community as many people are shy or just don't say much sometimes.

I will respond more again, but one thing I will mention is that so far we've avoided having any bash scripts that are directly related to configuring SlickStack settings/permissions... this was originally because I worried it would degrade the stability of the stack like if certain cron jobs were not running etc. By integrating SlickStack settings/permissions directly into the cron job templates and other module bash scripts, I thought it would help avoid confusion and single points of failure.

Re: what settings the pilot file should contain, that is something I've been hoping to hear feedback on... generally speaking I think it would have to avoid sensitive fields like passwords since it's being updated via wget and potentially insecure mirror locations etc... although, ironically, the sudo password might be the most common setting that many developers wish to quickly change across 100+ cloud servers at the same time...

I'm also starting to think we might need another ss-worker type of script that runs less often, because some of the tasks that we have it doing these days should not be running as often as they are running.

[backamblock]

for the ss-worker problem: i don't understand the why behind every function and why it runs at that time yet, so i really have no good opinion on that this time.

I guess it would be enough to check the pilot file hourly at most, things in there should not change on a regular basis.
for the passwords, that is exactly what i personally would not put in there.
this would be better off in a little ssh script that runs locally imho. it reads a csv with "IP","USER","OLD_PASSWORD","NEW_PASSWORD" and ssh in the machines to change passwords. it should easily handle 100 changes per minute.

as a way to keep it in the pilot file: encrypted pastebins, if thrustworthy ones exist, or bitwarden encrypted "sends"
the last option can be protected with a password which would in turn be saved locally on the VPS beforehand

this is what the install side would look like. not finalized, just a draft at this point

for the login, i am only using ssh keys. the sudo password is only a backup for me in case laptop destroyed, ...

for the config the solution could be this, which is sent to some storage or email in encrypted form:

right now i am using this and scp it to my local machine after the install but i see no negative about sending it somewhere else.

[jessuppi]

https://slickstack.io/docs/what-is-a-pilot-file