Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
NixOS: declarative linux distribution
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Sun Mar 24, 2013 6:07 pm    Post subject: NixOS: declarative linux distribution Reply with quote

Hi all,

as a long time Gentoo user I would like to point out NixOS, Linux distribution that changes a lot about how we handle packaging today in Linux.

I'd first like to point out what it does better and what it does worse than Gentoo:

Pros:

[*] binary packages: if available, otherwise it compiles from source
[*] declarative configuration: repeatable system configuration (see below how it works)
[*] I can configure the whole system in a virtualmachine and when it's ready, push new configuration to production
[*] rollbacks: NixOS keeps a history of all changes and when you actually upgrade, just symlinks are changed to new software/configuration folders. This allows rollbacks to any previous configuration (except if you garbage collect them to spare HDD space)
[*] tests: NixOS has an idea of tests, since it repeatably installs software and it's configuration, it can also test the state of result
[*] hydra: this is the best part. If you change for example gcc installation proceedure, hash of gcc will change and everything that depends on gcc as runtime dependency, will be recompiled on hydra continous integration server and new binaries will be provided

Cons:

[*] small community -> we are about 50-100 people and a lot of software is packaged due to our easy workflow, but we need more users!
[*] stability -> it's far less stable than gentoo mostly because there are no standards yet how to test rolling releases
[*] nix is a DSL that you have to learn -> it's simple, but it takes about 2 days to read syntax and being able to write it
[*] it compiles full blown binaries -> you can make dependencies optional, similar to USE flags in gentoo, but by default most software is compiled as full featured

I don't want to go deep inside how it all works, but in short, there are four parts:

Nix:

Very slim functional language. It tries to separate installed software into folders and symlink them together for a functioning system.
The main goal is to isolate everything when building software. There are some impurities, but it's isolating most that can be done.

A folder looks like:

Code:
/nix/store/r8vvq9kq18pz08v249h8my6r9vs7s0n3-firefox-2.0.0.1/


In that folder, Firefox is installed as prefix to the folder. It's dependencies are also folders in /nix/store/. The idea is that sha hash of the firefox is a hash of all inputs - including dependencies.

For Nix syntax, see: http://hydra.nixos.org/build/4462862/download/1/manual/#chap-writing-nix-expressions

nixpkgs:

Nixpkgs is a collection of packaged software. For example, Vim: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/default.nix

See phases of nixpkgs: http://hydra.nixos.org/build/4465618/download/2/manual/#ssec-stdenv-phases

NixOS:

Declarative configuration for nixpkgs. It practically renders configuration to the filesystem based on one config file, so installation of the whole system is repeatable.

Packages are installed from Hydra as binaries (if available), otherwise they are compiled.

Example of configuration.nix:

Code:

{
  boot.loader.grub.device = "/dev/sda";
 
  fileSystems = [{
    mountPoint = "/";
    device = "/dev/sda1";
  }];

  services.sshd.enable = true;
}


Hydra

This is the best part. It's a server that builds nixos/nixpkgs based on git commits and the hashes that change. It provides binary files for certain architectures for NixOS install.


To try it yourself: http://hydra.nixos.org/build/4465371/download/1/nixos/manual.html


Last edited by iElectric on Sat Sep 28, 2013 3:40 pm; edited 1 time in total
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Mon Mar 25, 2013 6:13 am    Post subject: Reply with quote

Interesting.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 318
Location: NRW, Germany

PostPosted: Mon Mar 25, 2013 10:49 am    Post subject: Reply with quote

Mh.
You talked a lot about what NixOS does. I'd be much more interested in the "why".
Back to top
View user's profile Send private message
fragment
n00b
n00b


Joined: 08 Mar 2013
Posts: 13

PostPosted: Mon Mar 25, 2013 5:06 pm    Post subject: Re: NixOS Reply with quote

iElectric wrote:

[*] rollbacks: NixOS keeps a history of all changes and when you actually upgrade, just symlinks are changed to new software/configuration folders. This allows rollbacks to any previous configuration (except if you garbage collect them to spare HDD space)


Why is it better than snapshots (on BtrFS, for example)?
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Mon Mar 25, 2013 10:10 pm    Post subject: Reply with quote

Dr.Willy wrote:
Mh.
You talked a lot about what NixOS does. I'd be much more interested in the "why".


Not sure what you mean by "why".
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Mon Mar 25, 2013 10:14 pm    Post subject: Re: NixOS Reply with quote

fragment wrote:
iElectric wrote:

[*] rollbacks: NixOS keeps a history of all changes and when you actually upgrade, just symlinks are changed to new software/configuration folders. This allows rollbacks to any previous configuration (except if you garbage collect them to spare HDD space)


Why is it better than snapshots (on BtrFS, for example)?


I'd say because:

[*] Nix doesn't require BtrFS (yes, I'd call that a feature)
[*] Nix keeps a history of changes for every change, so you don't need to do snapshots, it's a builtin feature. When you want to purge history, you can delete everything older than x days.
[*] Nix has "profiles" so that multiple sets of packages can be installed at the same time from Nix store, even by different users (Nix store is readonly)

Those are basically all connected to the fact that Nix understands "snapshots", it's not a feature of the filesystem.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 318
Location: NRW, Germany

PostPosted: Mon Mar 25, 2013 11:15 pm    Post subject: Reply with quote

iElectric wrote:
Dr.Willy wrote:
Mh.
You talked a lot about what NixOS does. I'd be much more interested in the "why".


Not sure what you mean by "why".

What I mean by "why"?
What I mean is: What is the fucking point?
Why do I want to isolate everything when I compile a package? (And what am I isolating anyway?)
Why would I want to hash every input of firefox and put it in front of the folder name?
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16090
Location: Colorado

PostPosted: Mon Mar 25, 2013 11:33 pm    Post subject: Reply with quote

BoneKracker wrote:
Interesting.
++

I like the binary / source combination. A feature I've often thought would be useful with Gentoo.

My main hesitation would be yet-another-distro syndrome. I'm not entirely sure it solves a problem, rather than say extending an existing distro.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Tue Mar 26, 2013 1:11 am    Post subject: Reply with quote

There are several interesting things here, from an administration perspective, particularly in terms of rapid provisioning and/or use as a virtual appliance. The things that look "different" to me have all been done before in some context, but this is a unique combination:

1. Centralized, standardized configuration of everything; this is the Windows Registry or that Gnomish freedeskfap.org equivalent. I like the object-oriented notation as opposed to some kind of unreadable XML-ish markup. This should facilitate rapid provisioning, use as a virtual appliance, standardized "control panel" style configuration interface, etc. It also presents a common point of failure and an opportunity for cross-pollution and mutual butchery.

2. Hydra: Server-based configurator and automated build system? That would automate some package maintenance, facilitate use as a meta-distribution, and ease provisioning. It might be useful even at individual and small organization level.

3. Application folders, with links to its contents throughout the standard linux file hierarchy. Steve Jobs wanted application installation to be as easy and dropping a folder and deinstallation just removing a folder. I think Apple, BeOS, and NeXT all tried this to one extent or another, and there's a linux distro out there that also does what's being described here (. I'm not sure I see the utility of it, given the need to put symlinks everywhere anyway. I suppose it makes it easy to identify orphans, by just looking for dead symlinks. But then you've still got "registry" entries to clean up.

I don't understand why you guys are talking about filesystem snapshots, I don't see anything here that's similar. The "rollbacks" he referred to look like the ability to undo installation and configuration changes. It doesn't appear to offer the ability to revert changes to application data or machine state.

I'd like to know more about how the centralized configuration file is managed (apparently it's a text file and not a database) and how it gets converted into configuration information used by each application (and vice-versa, if I user makes a change to the application configuration that would normally be recorded in a config file, how does that make it back into the centralized configuration file).

I'd also like to know about compile-time configuration versus run-time configuration. Is the first kept in "Hydra" and the latter kept in /etc/nixos/? Or do you manage most of the run-time configs through hydra as well, by setting them as default values?
Back to top
View user's profile Send private message
fragment
n00b
n00b


Joined: 08 Mar 2013
Posts: 13

PostPosted: Tue Mar 26, 2013 3:53 am    Post subject: Re: NixOS Reply with quote

iElectric wrote:


[*] Nix doesn't require BtrFS (yes, I'd call that a feature)
[*] Nix keeps a history of changes for every change, so you don't need to do snapshots, it's a builtin feature. When you want to purge history, you can delete everything older than x days.
[*] Nix has "profiles" so that multiple sets of packages can be installed at the same time from Nix store, even by different users (Nix store is readonly)

Those are basically all connected to the fact that Nix understands "snapshots", it's not a feature of the filesystem.


It looks interesting. I guess, that easy to realize all of this features on usual Gentoo with BtrFS.

I think, that keeping of the history must to work on a file system level, not through some high-level crutches.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16090
Location: Colorado

PostPosted: Tue Mar 26, 2013 4:11 am    Post subject: Reply with quote

Seems to have features Gentoo does not, and the filesystem method seems to be more of the crutch IMO. Interesting how people can see things differently. (I'm a fan of ZFS, and hope someday btrfs will be viable).
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
fragment
n00b
n00b


Joined: 08 Mar 2013
Posts: 13

PostPosted: Tue Mar 26, 2013 5:08 am    Post subject: Reply with quote

pjp wrote:
(I'm a fan of ZFS, and hope someday btrfs will be viable).


What problems with BtrFS? :roll:
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Tue Mar 26, 2013 9:05 pm    Post subject: Reply with quote

Dr.Willy wrote:
iElectric wrote:
Dr.Willy wrote:
Mh.
You talked a lot about what NixOS does. I'd be much more interested in the "why".


Not sure what you mean by "why".

What I mean by "why"?
What I mean is: What is the fucking point?
Why do I want to isolate everything when I compile a package? (And what am I isolating anyway?)
Why would I want to hash every input of firefox and put it in front of the folder name?


For a very simple reason: repeatability.

Currently in gentoo, input to compile a package is current filesystem structure. Which may be a lot of things that present the input.

In Nix(OS), package is compiled only by a whitelisted inputs (being folders that presents packages). That means once you compile the package given isolated inputs, the output is always the same (on specific arch).

Hash of firefox is just an implementation detail how Nix detects if input has changed (and that means firefox or any of dependencies).
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Tue Mar 26, 2013 9:07 pm    Post subject: Reply with quote

pjp wrote:
BoneKracker wrote:
Interesting.
++

I like the binary / source combination. A feature I've often thought would be useful with Gentoo.

My main hesitation would be yet-another-distro syndrome. I'm not entirely sure it solves a problem, rather than say extending an existing distro.


Agreed. There always needs to be a very good "yet-another-distro" reason and I think for NixOS, there exists several (as stated in the first post).
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Tue Mar 26, 2013 9:16 pm    Post subject: Reply with quote

BoneKracker wrote:
There are several interesting things here, from an administration perspective, particularly in terms of rapid provisioning and/or use as a virtual appliance. The things that look "different" to me have all been done before in some context, but this is a unique combination:

1. Centralized, standardized configuration of everything; this is the Windows Registry or that Gnomish freedeskfap.org equivalent. I like the object-oriented notation as opposed to some kind of unreadable XML-ish markup. This should facilitate rapid provisioning, use as a virtual appliance, standardized "control panel" style configuration interface, etc. It also presents a common point of failure and an opportunity for cross-pollution and mutual butchery.

2. Hydra: Server-based configurator and automated build system? That would automate some package maintenance, facilitate use as a meta-distribution, and ease provisioning. It might be useful even at individual and small organization level.

3. Application folders, with links to its contents throughout the standard linux file hierarchy. Steve Jobs wanted application installation to be as easy and dropping a folder and deinstallation just removing a folder. I think Apple, BeOS, and NeXT all tried this to one extent or another, and there's a linux distro out there that also does what's being described here (. I'm not sure I see the utility of it, given the need to put symlinks everywhere anyway. I suppose it makes it easy to identify orphans, by just looking for dead symlinks. But then you've still got "registry" entries to clean up.

I don't understand why you guys are talking about filesystem snapshots, I don't see anything here that's similar. The "rollbacks" he referred to look like the ability to undo installation and configuration changes. It doesn't appear to offer the ability to revert changes to application data or machine state.

I'd like to know more about how the centralized configuration file is managed (apparently it's a text file and not a database) and how it gets converted into configuration information used by each application (and vice-versa, if I user makes a change to the application configuration that would normally be recorded in a config file, how does that make it back into the centralized configuration file).

I'd also like to know about compile-time configuration versus run-time configuration. Is the first kept in "Hydra" and the latter kept in /etc/nixos/? Or do you manage most of the run-time configs through hydra as well, by setting them as default values?


I you almost see the big picture :) Maybe read http://design.liberta.co.za/articles/customizing-disabling-vim-matching-parenthesis-highlighting/

Yes, NixOS is wonderful for testing in virtualmachine, it even has a parameter when upgrading that it builds a vm instead of upgrading current machine. Virtual appliances are one of things that are very repeatable with NixOS to extend that you can build them for every commit.

1. Yes. While windows registry or gnome registry is only a binary blob, nix is a language with lots of supported functions that come in handy. A nixos configuration has always two important parts:

options: which presents what options does the module provide for configuration and

config: which presents what module configures on system (renders configuration files, sets up new users, configures systemd services, ...)

2. Indeed, once a package has set "meta.platforms", nixos will build it, run tests and generate packages for those platforms.

3. You change settings only through NixOS. NixOS makes sure it configures everything. It's also one of drawbacks of Nixos: you have to forget good old fashioned Linux administration.

To do bidirectional mapping, tools like augeas need to be used that's a fuckedup area you don't want to try.

Run-time configuration: can you give an example?
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16090
Location: Colorado

PostPosted: Tue Mar 26, 2013 10:22 pm    Post subject: Reply with quote

iElectric wrote:
Agreed. There always needs to be a very good "yet-another-distro" reason and I think for NixOS, there exists several (as stated in the first post).
Most of what I read seemed like it could have been implemented by extending an existing distro. I'd probably have to use it for a while to get a better understanding. I will say it sounds better than many other distros I've read about in the recent past.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16090
Location: Colorado

PostPosted: Tue Mar 26, 2013 10:22 pm    Post subject: Reply with quote

fragment wrote:
What problems with BtrFS? :roll:
http://lmgtfy.com/

I haven't kept a close eye on it, but last I checked, it seemed there were still some data integrity and usability issues. This isn't the thread for a btrfs war.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Sat Mar 30, 2013 5:36 am    Post subject: Reply with quote

Okay, somebody go try this and let me know how it is.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 318
Location: NRW, Germany

PostPosted: Sat Mar 30, 2013 12:09 pm    Post subject: Reply with quote

BoneKracker wrote:
Okay, somebody go try this and let me know how it is.

Yeah, I'll be right behind you.
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Sat Mar 30, 2013 4:00 pm    Post subject: Reply with quote

You can try it real quick on virtualbox (or other vm): http://nixos.org/wiki/Installing_NixOS_in_a_VirtualBox_guest

I configure/update all my machines configuration in vm, then when tested, do a git push and update the "production" macihe.
Back to top
View user's profile Send private message
smartass
Apprentice
Apprentice


Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)

PostPosted: Sat Mar 30, 2013 5:45 pm    Post subject: Reply with quote

As a concept it looks very interesting and it's impressive what it can do.

However, I get the feeling that casual users aren't the target audience, because

1. it tries to reinvent the wheel, going against most common approaches, meaning documentation will be harder to find and some exotic packages counting on some standards will be hard to build
2. does not build upon an established base, so the package collection isn't that big
3. it appears to be designed more like a meta-distribution, an appliance generation system with continuous integration capabilities

So, I can imagine using it if I were building/maintaining an operating system/virtual appliances for a particular purpose in a cluster/cloud environment.
Otherwise I'm not sure I'd ever use all those interesting features on a scale that would convince me to invest time into learning how to use it.
Unless they improve on points 1. and 2., I don't see how they could attract more casual users.
Looking at the content of their package files, the functional content is very close to what most ebuilds contain.

If I were a NixOS guy and wanted to attract users, I'd start building upon Gentoo, extending the current EAPI and portage to use the technologies developed in NixOS. That would give a me a larger user,developer and tester base and I'd have an established base and years of experience to build upon.

Otherwise my guess is that NixOS will evolve into an upstart offering and alternative approach to current enterprise virtual appliance management software.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16090
Location: Colorado

PostPosted: Sat Mar 30, 2013 9:56 pm    Post subject: Reply with quote

smartass wrote:
As a concept it looks very interesting and it's impressive what it can do.

However, I get the feeling that casual users aren't the target audience, because

1. it tries to reinvent the wheel, going against most common approaches, meaning documentation will be harder to find and some exotic packages counting on some standards will be hard to build
2. does not build upon an established base, so the package collection isn't that big
3. it appears to be designed more like a meta-distribution, an appliance generation system with continuous integration capabilities

So, I can imagine using it if I were building/maintaining an operating system/virtual appliances for a particular purpose in a cluster/cloud environment.
Otherwise I'm not sure I'd ever use all those interesting features on a scale that would convince me to invest time into learning how to use it.
Unless they improve on points 1. and 2., I don't see how they could attract more casual users.
Looking at the content of their package files, the functional content is very close to what most ebuilds contain.

If I were a NixOS guy and wanted to attract users, I'd start building upon Gentoo, extending the current EAPI and portage to use the technologies developed in NixOS. That would give a me a larger user,developer and tester base and I'd have an established base and years of experience to build upon.

Otherwise my guess is that NixOS will evolve into an upstart offering and alternative approach to current enterprise virtual appliance management software.
Well put. Quantity of packages isn't as much of a concern to me as I think that will be solved over time if enough people have interest. But it very much seems like it should have piggy-backed on an existing distro. Gentoo or maybe Arch seem ideal.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Sat Mar 30, 2013 10:31 pm    Post subject: Reply with quote

I see no showstoppers why gentoo couldn't adopt that :)
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16090
Location: Colorado

PostPosted: Sun Mar 31, 2013 2:25 am    Post subject: Reply with quote

You're absolutely correct. Personally I don't see a justification for the learning curve of a new distro.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
iElectric
n00b
n00b


Joined: 25 Dec 2007
Posts: 19

PostPosted: Mon Apr 01, 2013 2:33 pm    Post subject: Reply with quote

smartass wrote:
As a concept it looks very interesting and it's impressive what it can do.

However, I get the feeling that casual users aren't the target audience, because

1. it tries to reinvent the wheel, going against most common approaches, meaning documentation will be harder to find and some exotic packages counting on some standards will be hard to build
2. does not build upon an established base, so the package collection isn't that big
3. it appears to be designed more like a meta-distribution, an appliance generation system with continuous integration capabilities

So, I can imagine using it if I were building/maintaining an operating system/virtual appliances for a particular purpose in a cluster/cloud environment.
Otherwise I'm not sure I'd ever use all those interesting features on a scale that would convince me to invest time into learning how to use it.
Unless they improve on points 1. and 2., I don't see how they could attract more casual users.
Looking at the content of their package files, the functional content is very close to what most ebuilds contain.

If I were a NixOS guy and wanted to attract users, I'd start building upon Gentoo, extending the current EAPI and portage to use the technologies developed in NixOS. That would give a me a larger user,developer and tester base and I'd have an established base and years of experience to build upon.

Otherwise my guess is that NixOS will evolve into an upstart offering and alternative approach to current enterprise virtual appliance management software.


You are missing the whole point, but I agree some decisions were not the best. If someone is looking for a prefect distro, this is not the intention :-)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum