Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Air-Gapped Gentoo Install, Tentative
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Wed Mar 26, 2014 5:44 pm    Post subject: Air-Gapped Gentoo Install, Tentative Reply with quote

EDIT START Sun Apr 20 21:28:35 BST 2014
As introduced here:
https://forums.gentoo.org/viewtopic-p-7539048.html#7538138
I think this thread should be renamed:
Air-Gapped Gentoo Install, Tentative
simply because that is what all this is about.
WARNING Pls. bear with me. My ideas weren't at all clear when I started this
thread. However, not out of brazenness, and if you skim faster through the
unclear parts in the beginning, you will notice that later on my understanding
of the matters starts to come into shape.
Thank you!
EDIT END

EDIT START Fri Mar 28 18:24:38 UTC 2014
The title was previously wrong:
Offline Install, use emerge-webrsync to check and log?
Pls. see here:
https://forums.gentoo.org/viewtopic-t-987268.html#7525726
why that was wrong... Sorry again. Consistently with the wrong title, lots of my
understanding was unclear and plain wrong, when I opened this topic... Clearing out slowly...
EDIT END

Offline Install, how to use emerge-webrsync to check and log every package in
the distfiles?

Well, at least check and log them as they are installed.

(( to some extent, I am continuing on some issues from:
https://forums.gentoo.org/viewtopic-t-984066.html ))

I've already collected a few packages, and I don't want to redownload them.
I'm not an expert to feel like a fish in the water online, and am aware how
little it takes to break into systems, for experts... My main defence is
having a clean backup, reverting to when things were clean. dd dumps are
images to the bit of the device they dumped, and I know how to backup my
systems. I wrote already about it on Gentoo Forums, and will give the links
here, if I get less strapped with time, i.e. succeed in my reinstalling of
Gentoo for one of my boxes, which then I will easily clone onto other of my
systems, as I have a few same MBO, similar hardware boxes...

My idea is to use emerge-webrsync to check packages...

I couldn't easily come to terms with the explanations in the Handbook (we're
talking AMD Handbook here

http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1
at the time of writing this post

) on emerge-webrsync, and it took me time and some searching to figure out
some of, but not all of these issues. Namely that if you use emerge-webrsync
then you don't do any more of emerge --sync ... But, to be able to do that,
proper configuration is needed.

I found somewhere that putting into /etc/portage/make.conf:

SYNC=""

that is, an empty string, would disable the rsyncing but am yet to learn if it
will really do so for me. I guess it will.

After deploying the stage3 tar ball, somewhere around here in the Handbook:

( Pulling Validated Portage Tree Snapshots )
http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#webrsync-gpg

those:

Code:

sync-type = rsync
sync-url


that are mentioned in that section, can only be found in:

/usr/share/portage/config/repos.conf

so I don't think that needs commenting out, but somebody correct me if I'm
wrong.

Also, it took me a while to figure out these changes should accomodate better
my needs. I mean, I like the emerge-webrsync to keep what it downloads, and
I'd like it to be, oh, so much more talkative...

I'm writing offline... (a preemptive remark: look up this link if anyone
considers that paranoid, attacks on my machines are documented and
undeniable...:

grsec: halting the system due to suspicious kernel crash
http://forums.grsecurity.net/viewtopic.php?f=3&t=3709

)

I'm not writing from LiveCD Box, but from sysresccd running from RAM, but I
can copy my sed lines that I intend to run on /usr/bin/emerge-webrsync before
I use it, by hand:

Code:

sed -i.bak 's/do_verbose=0/do_verbose=1/' | sed 's/keep=false/keep=true/'


For the less initiated that would give you a /usr/bin/emerge-webrsync that
has:
do_verbose=1 instead of do_verbose=0
and
keep=true instead of keep=false

and it'll hold those till the next upgrade of itself... (as that is not a conf file)

But it is not sufficient for my needs to just make it verbose by default and
make it keep the portage snapshot by default...

I have already tried and failed in installing, with some strange errors, and I
can't tell whether I did someting wrong or other reasons were for the
failure... I don't keep logs of everything, I just remember that it looked a
little suspicious, and so...

I don't keep logs of everything, esp. when I can't do so... I like when it is
possible to do so, and would like to see how much of the logs I could possibly
get, on the verification of the packages. Such as, I like the logs that I can
get with Grsecurity, they often tell interesting stories, although more to
experts than me. Again, look up the link on Grsecurity Forums I gave above,
where Grsecurity hardened Gentoo shines just fine, defeating intrusion in my
systems, to some extent.

Isn't it useful to users and developers, having such logs to report?

Let me give you my plan and a few ideas, I hope if I get good advice, this
could be useful to others (or am I the only one having hard time with
surveillance? ;-) )

What I intend to do, as well as what I have already done (rewriting this for
an umptiethe time) is as follows.

Boot with Gentoo official LiveCD.

Set the time to the right hour in the past when the portage that I kept with

Code:
(chroot) livecd # emerge-webrsync -k


was first deployed (at the time of running that command, deployed by the run
of that command). I want to do so, lest it don't complain of wrong timing
(that's non-intrinsic, but may prove the right thing to do)...

The portage snapshots are (with the -k given) kept in /usr/portage/distfiles/
such as:
/usr/portage/distfiles/portage-20140323.tar.xz
/usr/portage/distfiles/portage-20140323.tar.xz.gpgsig
/usr/portage/distfiles/portage-20140323.tar.xz.md5sum

What I did is:
followed the Handbook up to unpacking the stage3 tarball, and some way
further, somewhere around connecting to the internet, but instead of
connecting to internet this time around, using this time that which was
downloaded the last time.

First the portage snapshot:

Code:
cd /usr

tar xJvf portage-20140323.tar.xz


Of course it is
tar xJvf /somewhere-where-I-stored-it/portage-20140323.tar.xz

and there, the portage tree is installed, complete, and what is important,
trustful, trustful so far.

But here comes the challenge. How do I do the next step?

It doesn't have to be as tedious as is threatened ;-) to be, here:

https://wiki.gentoo.org/wiki/FAQ#I_have_only_slow_modem_connection_at_home._Can_I_download_sources_somewhere_else_and_add_them_to_my_system.3F

Quote:

...[snip]...
Put the sources into /usr/portage/distfiles/ and then simply run emerge
package. Be warned, however, that this is a tedious process.


Namely HDDs, are not so very expensive if they're not the latest huge ones,
and I can easily zero some of my HDD, and apply the same GPT table as
previously, mke2tfs the partitions and such...

But I want to be able to do more than is mentioned in that FAQ.

The packages I have already collected, they were downloaded according to what
use flags I set into make.conf, they would have to be fine if I were to run
the same command as the last time, to emerge those same packages, wouldn't
they?

But I want to be able to check them with emerge-webrsync, and I would like to
log every single package as it is being checked.

This:
Code:
# equery b emerge-webrsync


will reveal to you that emerge-webrsync is part of portage package.

I see there no special flags on emerge-webrsync if I run:

Code:
# emerge -pvt portage


Also:
Code:
# emerge-webrsync -h

gives very scant information.

I found no special tutorial on emerge-webrsync on the Wiki or in the Forums...

I want to be certain that what I install from this point on is only that which
is signed with Gentoo signatures.

I don't mind having to do used zeroed HDDs to recreate the existing systems
from backup, I want to get cloneable privacy-viable Gentoo installation for my
machines on my SOHO at any cost. Other then "good good bullsh*t" cost (Pink
Floyd, 1970s I believe, "Money"). That I don't have. The Regime currently in
power in my country ruined my investments and I am poor.

So I want to be certain of all and any packages that I install from this point
on. I know there is no absolute certainty. But currently I have almost no
certainty at all...

I don't want any rogue packages, and since it is so easy for experts to break
into systems, within fractions of a second once you're online, a program ready
for you can play at least a few tricks on your system, can't it, especially
since GNU/Linux has long been disregarding security wholesale... few
exceptions there...

This is not easy what I want, is it?

Any ideas?

I can read bash code (emerge-webrsync is in bash), but I take soo loong to
understand it, so much research...

If I don't get a quicker advice, I'll probably be back but not very soon...

I might also be off for a few hours starting at imprecise time soon from now,
for unrelated other obligations I have. But I will be back, God willing. Pls.
bear that in mind if anyone replies here. Thank you!

Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr


Last edited by miroR on Sun Apr 20, 2014 8:37 pm; edited 5 times in total
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Fri Mar 28, 2014 8:41 am    Post subject: Re: Offline Install, use emerge-webrsync to check and log? Reply with quote

miroR wrote:

I couldn't easily come to terms with the explanations in the Handbook (we're
talking AMD Handbook here

http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1
at the time of writing this post

) on emerge-webrsync, and it took me time and some searching to figure out
some of, but not all of these issues. Namely that if you use emerge-webrsync
then you don't do any more of emerge --sync ... But, to be able to do that,
proper configuration is needed.

I found somewhere that putting into /etc/portage/make.conf:

SYNC=""


Pls. have a look at how the following comply with each other.

/mnt/gentoo/usr/lib/portage/bin/emerge-webrsync
on my so far deployed Gentoo which I started installing, but also the same can
be found on
/usr/lib/portage/bin/emerge-webrsync on any current system with
regularly emerged emerge-webrsync

The script is 524 lines, and lines 505-513 are (manual copy, for reasons given
above):

Code:

if [[ -n ${repo_sync_type} && ${repo_sync_type} != rsync ]] ; then
echo "The current sync-type attribute of repository 'gentoo' is not set to
'rsync':" >&2
echo >&2
echo " sync-type=${repo_sync_type}" >&2
echo >&2
echo "If you intend to use emerge-webrsync then please" >&2
echo "adjust sync-type and sync-uri attributes to refer to rsync" >&2
echo "emerge-webrsync exiting due to abnormal sync-type setting" >&2
exit 1
fi



The above, and the following:

http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part2_chap3__chap6
3.f. Pulling Validated Portage Tree Snapshots
...[snip]...
Code:

Code Listing 6.3: Updating repos.conf
# Make sure sync-type and sync-uri are commented out
# sync-type = rsync
# sync-uri = ...


In the code in the emerge-webrsync says do it, and in the book says don't do
it...
Or is there something I am missing here?
It must be only an apparent conflict. The repos.conf is something
non-developers don't even use, do they?

Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr
EDITED Fri Mar 28 09:28:39 UTC 2014, replaced the incomplete lines from emerge-webrsync 509-512 with full if statement copy


Last edited by miroR on Fri Mar 28, 2014 9:31 am; edited 1 time in total
Back to top
View user's profile Send private message
TomWij
Developer
Developer


Joined: 04 Jul 2012
Posts: 1551

PostPosted: Fri Mar 28, 2014 9:22 am    Post subject: Reply with quote

SYNC is the old deprecated way, repos.conf is the new way; see https://forums.gentoo.org/viewtopic-t-969972-start-0.html
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Fri Mar 28, 2014 9:34 am    Post subject: Reply with quote

TomWij wrote:
SYNC is the old deprecated way, repos.conf is the new way; see https://forums.gentoo.org/viewtopic-t-969972-start-0.html

Thanks!
I sure am already looking into it :P
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Fri Mar 28, 2014 1:47 pm    Post subject: Reply with quote

I'm not completely certain, and so I'll watch be around to revert it, but I made changes to:

https://wiki.gentoo.org/wiki/Mirrorselect

I replaced the deprecated:

Code:
mirrorselect -i -r -o >> /etc/portage/make.conf


with the new:

Code:
mirrorselect -i -r -o >> /etc/portage/repos.conf


IIUC, the same should be done in the XML doc:

http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap6__chap1_sect1

but that is different kind of editing, wouldn't know how to do it now:
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Fri Mar 28, 2014 2:33 pm    Post subject: Reply with quote

miroR wrote:
I'm not completely certain, and so I'll watch be around to revert it, but I made changes to:

https://wiki.gentoo.org/wiki/Mirrorselect

I replaced the deprecated:

Code:
mirrorselect -i -r -o >> /etc/portage/make.conf


with the new:

Code:
mirrorselect -i -r -o >> /etc/portage/repos.conf


IIUC, the same should be done in the XML doc:

http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap6__chap1_sect1

but that is different kind of editing, wouldn't know how to do it now:


Ermmhh. It's a mess.
There is, and I thought at first that was correct:

https://wiki.gentoo.org/wiki/SYNC

such a line there.

But looking at man 5 portage, the repos.conf section, repos.conf does not have any SYNC variable, but sync-type and sync-uri.

Also man mirrorselect mentions no repos.conf

So I don't know.

Anyway, with emerge-webrsync, as I mentioned I would try and do checking on stout and logged of all the packages that are being installed, downloaded earlier, I think it must be possible to do it by either changing the source in the way I explained above, or making sure not to forget the -k -v flags when I run it...

I think I'll try and run

Code:
(chroot) livecd $ emerge-webrsync -k -v 2>&1 >> /somewhere/emerge-webrsync_check-n-log-pkgs.log


Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Fri Mar 28, 2014 6:22 pm    Post subject: Reply with quote

miroR wrote:

I think I'll try and run
Code:
(chroot) livecd $ emerge-webrsync -k -v 2>&1 >> /somewhere/emerge-webrsync_check-n-log-pkgs.log


Wrong. But I think it's slowly dawning on me which way to check packages and make trustworthy and safe install.

The emerge-webrsync download is safe already.
And how I deployed it is correct.

But the emerge-webrsync downloads are the equivalent of of just the ebuilds and things, not the tar.bz2 and such files with the acutal programs. Sorry for lay terms I'm using. Don't know better yet.

It's the "digest" in make.conf and in ebuild, and in repoman manpages, as well as emerge manpage itself, that I am studying now, and I believe I'm closer ti what I want.

Also I need to change the title of this topic, because it is misleading.
Sorry!
Back to top
View user's profile Send private message
vaxbrat
l33t
l33t


Joined: 05 Oct 2005
Posts: 620
Location: DC Burbs

PostPosted: Fri Mar 28, 2014 7:49 pm    Post subject: Use btrfs for snapshotting Reply with quote

You might look into using btrfs and its snapshot ability. Create subvolumes that you mount on /usr/portage, /var/db/pkg and /usr/portage/distfiles. Take a snapshot before each emerge activity and then you can do a directory comparison between the snapshot and the subvolume after the activity has finished. The snapshotting is almost instantaneous since it only copies extent pointers and thus doesn't take much additional storage either.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Sat Mar 29, 2014 12:34 pm    Post subject: Reply with quote

vaxbrat wrote:
You might look into using btrfs and its snapshot ability. Create subvolumes that you mount on /usr/portage, /var/db/pkg and /usr/portage/distfiles. Take a snapshot before each emerge activity and then you can do a directory comparison between the snapshot and the subvolume after the activity has finished. The snapshotting is almost instantaneous since it only copies extent pointers and thus doesn't take much additional storage either.


It's not the snapshots that would help in safety, but only in speed.

And it's not the few dumping and restoring that aches here, that's relatively little.

It's how to evaluate the already downloaded distfiles, because
emerge-webrsync's portage is safe already.

It is the way of the digest, how to check every package by its manifest in the
ebuild.

I'll go slowly. Package by package, to learn how to manually do it, or which
command to use, on a per package basis.

I want a privacy-viable Gentoo, and the only option in my view, is Grsecurity
Hardened, but on a system that has all the packages checked for consistency,
right from the beginning of the instlall.

I am no fish for that water, to be able to stay online and emerge things and
feel safe anymore, for the reasons I explained above, giving links to actual
attacks on my system that I suffered, documented, and I feel this is the only way.

But it will be time consuming, since this certainly requires some advaced skills...

Anyway, vaxbrat, surely thank you for caring!
Miroslav Rovis
www.CroatiaFidelis.hr
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Mon Mar 31, 2014 9:20 pm    Post subject: Reply with quote

I still hope I can do a safe offline Gentoo install.

Similarly to the method of installing Debian on now-unacceptably-slow-for-
Gentoo-style-compilations old machines of mine, completely offline. Because
I did manage to achieve safe and secure offline install there. My Debian
systems, once cloned from the master offline-installed box, now break only
once they go online, after some time online, that is, after exposure to
intrusions which in my case, a homeland-living strong dissenter and critic
( 460d21cf0df780a9652d95baaa5f779b (note *1*) )
of the Regime of neo-commie bloodthirsty traitors' progenie criminal crony
capitalists in power in Croatia, which in my case is almost guaranteed.

But I was speaking about the method that I posted about on Tips and Tricks
pages on Debian Forums. Here:

How to Install Debian Offline from Your Local Mirror
http://forums.debian.net/viewtopic.php?f=16&t=111904

Of course, it'd be comparing "apples and eggs", comparing Gentoo and Debian

https://forums.gentoo.org/viewtopic-t-971348.html#7409042

and I don't intend to make a comparison in anything but how the distribution
is fetched for one and for the other of the two. Any other comparison btwn
these two would make little sense if any. The differences btwn these GNU/Linux
flavors are huge in almost any aspect.

To install Debian offline, is actually very easy, and with such safe install
evade any potential attack, any intrusion, in fact almost potentially any
surveillance whatsoever, sure only while offline, through the complete
non-exposure to the internet of such system. If you are doubtful or curious
of the truthfulness of that statement, you can validate it for yourself by
reading through the links I gave.

But Debian is OTOH certainly less reliable. They don't even digitally sign the
checksums of the ISOs containing their weekly bleeding edge distro! I'm not
talking behind their backs, I wrote about that openly on Debian Forums (I
did't find it in a quick search, but maybe somewhere here:

Grsecurity/Pax installation on Debian GNU/Linux
http://forums.debian.net/viewtopic.php?f=16&t=108616

)

Why did I mention the method of Debian offline install? What comparison is
there to make?

Because I think I can achieve a similar offline install with Gentoo, as with
Debian, in the sense that there exist such options, similar in result only,
to get all the package distfiles, all the distribution sources, and have
emerge use the local mirror for installation.

Code:


# man emirrordist

EMIRRORDIST(1)                      Portage                     EMIRRORDIST(1)



NAME
       emirrordist - a fetch tool for mirroring of package distfiles


...[snip]...
       


My question here is, can I get all the packages (or would I maybe anyway get
only those), for just my arch, with this command? How?

I actually reread just now that manpage, and it's a few more things that are
not at all clear, such as 'whitelist', such as where do I find what can go
into EMIRRORDIST_DEFAULT_OPTS, presumably in make.conf, and other things.

I read, by now, a few times, with increased understanding every next time, but
still far from complete understanding certainly, the fundamental reading
manuals of Gentoo:

# man make.conf
# man emerge
# man portage
# man 5 ebuild
# man 1 ebuild

and others...

The local mirror is talked about in man 5 portage, in the section repos.conf,
subsection sync-uri, not in Sysresccd Gentoo docs (still browsing with it), so
manually copying:

Code:

sync-uri
    Specifies URI of repository used for synchronization performed by `emerge
    --sync'.
    This attribute can be set to empty value to disable synchronization of
    given repository. Empty value is default. (note *2*)

    Syntax:
        cvs: ...[snip]...
   git: ...[snip]...
   rsync:    (rsync|ssh)://[username@]hostname[:port]/path

    Examples:
        rsync://private-mirror.com/portage-module
   rsync://rsync-user@private-mirror.com:873/gentoo-portage
   ssh://ssh-user@192.168.0.1:22/usr/portage
   ssh://ssh-user@192.168.0.1:22/\${HOME}/portage-storage

    ...[snip]...


I understand those in bottom are private-only adresses, such as, among other
purposes, for a SOHO like mine. And that is what eventually the emerge on my
some-time-not-long-from-now-in-the-future install should be drawing it's
package distfiles from... I feel I undestand closer that part... I've managed
my SOHO for quite a number of years.

Anyway, it's not just my asking for help that I post this topic here for, but
I hope it's also going to be useful for others.

I remember I heard on Russia Today, months ago now, which channel since their
agression on Ukraine's Crimea...

It's in the link already given above, along with the title of:
How to Install Debian Offline from Your Local Mirror
where I wrote also in support of Ukraine, find:

Code:

Спава Укrаине!

Glory to Ukraine!, Yanukovich out!

And that was before he was ousted, ousted by the kind of people my friends
are, who love their country above their own lives.

I lost almost any taste anymore for watching RT, Russians are really a
delusion to me, I feel compassion and pray that the American Edward Snowden
will not have hard times now that he is confined to live in Russia, for just
being American, and am proud of Russians like Khodarkovsky and like the guy,
Maxim Kamerer, IIRC, who made the great Liberté Linux, based on
Grsecurity-hardened Gentoo, but strong sanctions on Russia should be
imposed... And so I also approve of many packages having LINGUA="-ru" set.

But I know from exactly Russia Today, months ago they reported that Guardian
experts suggested no use of any of your really private data on a computer that
you are connected to internet with. And not to connect a computer that should
be private to any online computer in such way as wire or wireless, but the
data to say post publically, once you decide them out of some private stash,
transfer phisically into the online computer, such as with DVDs or USB sticks.

And so, what else, but complete offline install is in the order of the day, if
you simply care to not be surveilled?

I mean, if your system is backdoored right from the start, what chance do you
have of any privacy?

And there is no such Constitution of no such Democratic country, especially
not in the West, that does not guarrantee their own citizens freedom and
privacy!

I still hope, no I'm not overly confident, but I still hope that Gentoo will
remain privacy viable, and I believe it still is.

So I truly believe more of Gentoo users should try and think about this,
because Gentoo is GNU, and GNU may have lost some of its shine through the
years, but GNU is still the fundament of freedom, of good programs that belong
to all the good ( note *3*) people of the world.

GNU is still the licence that keeps GNU/Linux free, secure (surely only in the
Grsecurity way, not any NSA SELinux way. I mean, could anyone really ever
trust any spy agency for their privacy?), and I believe that Gentoo will
remain, but only through that freedom and that real security which
Grsecurity/Pax patched kernel is: privacy-viable.

GNU/Linux is really the only truly free and potentially surveillance-free
option in the world of computing. Apple? Microsoft? Google?... Heh, heh!
C'mon!

So, before I post the notes for *1* *2* and *3* above, more work on me, or not
so much if I find answers in my search of www.gentoo.org, and I guess, the
most of the work is now clearly cut: understand that difficult emirrordist, as
well as, I forgot to mention, maybe the '-F' option, that's uppercase F from
emerge itself...

Miroslav Rovis,
Zagreb, Croatia
www.CroatiaFidelis.hr
=====================

note *1* to be expanded later
EDIT START Tue Apr 1 11:30:15 UTC 2014
Here's some more hints:
Really Happened? 15e5510744048dc5473d05bfc028fbc2
https://forums.gentoo.org/viewtopic-p-7527616.html#7527616
EDIT END

EDIT START Tue Apr 1 20:41:59 UTC 2014
If at the time you try and click the link above you don't get a post that is readable,
i.e. if you see "links are disabled in the dustbin" instead of links, than see here:
https://forums.gentoo.org/viewtopic-p-7527914.html#7527694
EDIT END

note *2* So what I found somewhere on www.gentoo.org (surely not on official
pages), is wrong.

From:
https://forums.gentoo.org/viewtopic-t-987268.html#7524170
which is this topic, start of it, this:

miroR wrote:
I found somewhere that putting into /etc/portage/make.conf:

SYNC=""

that is, an empty string, would disable the rsyncing but am yet to learn if it
will really do so for me. I guess it will.


was all wrong!

note *3* ...and sufficiently knowledgeable, let's face it, GNU/Linux isn't easy, I
mean real GNU/Linux, not the commercialized backdoored flavors
==================
783bfc8aecba5dca95aa71d79f15fa4c


Last edited by miroR on Tue Apr 01, 2014 8:45 pm; edited 2 times in total
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Tue Apr 01, 2014 11:11 am    Post subject: Reply with quote

Here:
Project:Infrastructure/Mirrors/Distfile Mirroring System
http://wiki.gentoo.org/wiki/Project:Infrastructure/Mirrors/Distfile_Mirroring_System
I found some answers on the whitelists I mentioned in my previous post.

But it seems, reading there, that the emirrordist:
http://wiki.gentoo.org/wiki/Project:Infrastructure/Mirrors/Distfile_Mirroring_System#master_private_distfile_mirror
is for some other purposes, much more advanced.

And that this:
Project:Infrastructure/Rsync
http://wiki.gentoo.org/wiki/Project:Infrastructure/Rsync
is where I see how to set up my own local (rsync) mirror.

http://wiki.gentoo.org/wiki/Project:Infrastructure/Rsync#Setting_up_a_community_rsync_server

But that's not completely what I want, at all. Because I very much like the
emerge-webrsync because it is strongly digitally verifiable, and while no
movement in the world there existed where in which traitors never ever came in
from the outside, or outright grew to betray from in between the very ranks, I
hope I can still trust the Gentoo teams who sign the daily snapshot, and for
me, the portage snapshot is a great substitution for any other syncing. This
last statement is IIUC. Somobody correct me it I'm wrong.

Besides, the /etc/rsyncd.conf has:

Code:
exclude=distfiles/ packages/


It's how to get, and then in a routine of reasonable periods from there on,
update the distfiles/, in a way that is safe, which is the cause of my concern
and headache since I opened this topic that you are reading.

So that article about local rsync mirror is no solution for my headache there
at all.

This:
/etc/portage/mirrors
http://wiki.gentoo.org/wiki//etc/portage/mirrors
containing:
Code:

# local private mirror used only by my company
local ftp://192.168.0.3/distfiles

looks like somewhere where the distfiles can be used on a SOHO from centralized.

So I'm inching closer.

And I posted on rsync:Talk

https://wiki.gentoo.org/wiki/Project_Talk:Infrastructure/Rsync

about it.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Wed Apr 02, 2014 2:09 pm    Post subject: Reply with quote

Here's a preliminary review how I'll try and install Gentoo safely offline. I
don't live and don't work only for me. GNU/Linux also has grown from a
sense of common good wired deep in it. No one is going to be able to gut
it out of it, regardless of attempts growing...

But I won't repeat how I reached to some of the points, just I will accept
help from more knowledgeable people.

1) follow regular (AMD64 in my case) guide, up to (point to more precisely
define later)
**important** but use the hardened stage3 tarball, not the regular

2) Use -kv switches with emerge-sync to keep the snapshots (and if I revisit
the installation later reset the time to the time appox of the snapshot's
download)

3) forget about SYNC="" or keep it empty (still not completely in the
clear), but create
/etc/portage/repos.conf (and put the commented sync-type and sync-uri
in it --still not clear what exactly to do here either)

4) try and download distfiles with:
Code:
emerge -F

if that would do the downloading that we need for ofline install, because it
looks so:
Code:

--fetch-all-uri (-F)
      Instead of doing any package building, just perform fetches for all packages (fetch everything
      in SRC_URI regardless of USE setting).


If that works, there should be a rather voluminous stash of distfiles/ of at least
maybe 20GB or more, I'm really guessing here. Debian 12 DVD testing branch
containing entire installation is some 40-50GB, for example.

Save all, dd-dump all.

5) Recreate some of the system. Basically going from the beginning, but this
time completely offline. Completely new disk, same all, or cloned. Do all same
as before, except what needs online connection.

6) Copy or move the distfiles to a system where it will be served from. How to
serve files with Apache server is out of the scope of my research here, guides
on it widely available in different places.

7) On the offline system, among other things (what needs precise mentioning?),
set:
/etc/portage/mirrors:
as explained, not very verbosely, in:

https://wiki.gentoo.org/wiki//etc/portage/mirrors

Code:

# local private mirror
local http://192.168.x.x/distfiles

which needs to be the exact address where the downloads are available from for
the SOHO, as in 6) above

8 ) do the proper configuration for checking and logging of all that will next be installed

With this I mean some of the options such as strict in the FEATURES in make.conf
It looks to me that emerge wouldn't anyway install a package whose sizes let alone
sums do not correspond with the hashes in the manifest.

The portage system, the ebuilds and things look pretty confident to me. We'll see if
I fathom how to use them to keep my master installation good out of which I can easily
clone systems that can go online, because they can be restored completely from the
master install...

We'll see if I fathom how to use porage and its mechanisma to keep my systems
reliably trusty.... (similar to the method that I use with Debian, as I demonstrated
in Debian forums, links in previous posts)

Lots of work still ahead...

Now these points above, contain some completely (by me) not yet tested
assumptions, and it's all new territory for me...

Anyone tried something like that?
Any ideas?
M.R.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Tue Apr 08, 2014 9:13 am    Post subject: Reply with quote

In short, I'm still on this, top priority.

Just, it took loong to get entire distfiles/, 180G, and I took care to not
overload any server, and how could I anyway, when although I pay the 5Mbs
internet access to iskon dot hr (that is a Croatian provider), they keep me at
640Kbs.

However, again in brief, I stumbled upon a montain and am mustering faith to
move it into the sea. Joking, this time.

Half joking, I mean. Because it's big as a mountain to me. It's stranglingly
big (and I don't care it's not called after the strangler snake, because it is
strangling me :twisted: ). It's called Python.

What I am trying to say, is, all is there, in the code, all the checking, all
the logs to be produced for a really safe system, but this is my first real
encounter with this strangler language...

And I'll probably have to revamp my bash too.

Because those manuals that I mentioned above (portage, emerge, make.conf ...),
none of those can you really come to terms with without looking into the code.

The /usr/lib/portage/pym/portage/checksum.py
openly says:
Code:

$ head -1 /usr/lib/portage/pym/portage/checksum.py
# checksum.py -- core Portage functionality
$

and it's all written in that strangling language.
And the strangler shamelessly claims:
"Python is easy to learn, powerful programming language..."
(I emerged the python-docs and that is a quote.)

Easy?

I don't know, I'll give it a few days if necessary, I'll study the docs, and
if I make it to start understanding a little faster than the slowest turtle on
Earth can run, I'll patiently keep at it.

Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Mon Apr 14, 2014 1:33 pm    Post subject: Reply with quote

I'll try and expose the striking discrepancy that I found upon updating my
local mirror last night.

BTW, once you have your local mirror, you are no longer such a burden to
community for your updating of it, it looks to me, because for an update after
a period of, if my recollection serves me well, more than 10 days it took
downloading of only less than 3 GB to be done.

But I did an rsync dry run on a mirror, like so:

Code:

 rsync -nav rsync://some-mirror/gentoo/distfiles/ distfiles/


where distfiles/ is my complete mirror (sure I'm cd'd in the
storage/partition/wherever where it is)

which showed me what it would download, which was a fraction to add to what I
already got from the first mirror creation that I explained before, and then
did the real run:

Code:

 rsync -av rsync://some-mirror/gentoo/distfiles/ distfiles/

which still took long, but because of my slow (I explained exactly why in the
previous post) connection, but it's still much less time 2.9 GB vs 180 GB
download.

However, I also decided to compare in the same fashion, with an rsync dry run,
how the new updated mirror would update one of two other clones of that mirror
(having spent days to download the mirror, I really don't want to lose all
that effort, it would feel bad losing your work when you are to blame, than
when you are innocent, as in me vs Google terminating my account of 5 ys work,
500+ videos, where I have the sole, but shiny, consolation of clear
conscience).

And I ran, cd'ing into one of the clone-mirrors, something to this effect:

Code:

 rsync -nav /my-local-updated-mirror/distfiles/ distfiles/ > rsync-nav_delta.txt


That gave me a list to be updated, and I cleaned it from introducing lines and
finishing lines, which are not names of files that need to be rsync'd over,
and ran:

Code:

for i in `cat rsync-nav_delta_CLEAN-LIST.txt` ; do ls -l \
   /my-local-updated-mirror/distfiles/$i distfiles/$i ; read FAKE ; done ;


(read FAKE is just a way for me to make the script wait till I see what it
did, and till I hit Enter for it to continue)

But then the script asked to overwrite some files. Didn't accept at the time.

Rather I took care to see which files and why.

And here is the files that would be overwritten, and also the new files that
overwrite the old without asking in a regular rsync run:

distfiles_CHEK_overwrite.txt:
Code:

GeoIPASNum.dat.gz
GeoIP.dat.gz
GeoIPv6.dat.gz
GeoLiteCity.dat.gz
GeoLiteCityv6.dat.gz
timestamp.dev-local
timestamp.mirmon
VirtualBox-4.3.10.tar.bz2


Of course, timetamp-whatever is not the problem.
But, really why does the GNU free world accept only Google and Oracle
(I see Larry's mark on www.VirtualBox.org) forgo the naming conventions, and
esp. in the way that in this case two actually different versions of
VirtualBox-4.3.10.tar.bz2 have same version names?

Because I checked (will report if I find it to be different in the third of
the three different mirror archives, but I doubt it, read on why) and both the
"old" and the "new" VirtualBox-4.3.10.tar.bz2 unpack faultlessly, and would
probably install faultlessly (I don't want to use anything Oracle ever, if I
don't have to, but this is a matter of GNU principle).

Code:

mybox somewhere # for i in `cat  distfiles_CHEK_overwrite.txt` ; do ls -l distfiles/$i /mnt/sde1/distfiles/$i ; done ;
-rw-r--r-- 1 miro miro 1938996 Feb 17 17:20 distfiles/GeoIPASNum.dat.gz
-rw-r--r-- 1 miro miro 1947575 Apr  3 04:09 /mnt/sde1/distfiles/GeoIPASNum.dat.gz
-rw-r--r-- 1 miro miro 353106 Feb  5 16:59 distfiles/GeoIP.dat.gz
-rw-r--r-- 1 miro miro 383542 Apr  2 20:17 /mnt/sde1/distfiles/GeoIP.dat.gz
-rw-r--r-- 1 miro miro 560268 Feb  5 16:59 distfiles/GeoIPv6.dat.gz
-rw-r--r-- 1 miro miro 597350 Apr  2 20:17 /mnt/sde1/distfiles/GeoIPv6.dat.gz
-rw-r--r-- 1 miro miro 11049198 Feb  5 16:55 distfiles/GeoLiteCity.dat.gz
-rw-r--r-- 1 miro miro 10636449 Apr  2 20:49 /mnt/sde1/distfiles/GeoLiteCity.dat.gz
-rw-r--r-- 1 miro miro 11263430 Feb  5 16:49 distfiles/GeoLiteCityv6.dat.gz
-rw-r--r-- 1 miro miro 10854343 Apr  2 20:01 /mnt/sde1/distfiles/GeoLiteCityv6.dat.gz
-rw-r--r-- 1 miro miro 49 Apr  6 07:00 distfiles/timestamp.dev-local
-rw-r--r-- 1 miro miro 49 Apr 13 20:00 /mnt/sde1/distfiles/timestamp.dev-local
-rw-r--r-- 1 miro miro 11 Apr  6 07:53 distfiles/timestamp.mirmon
-rw-r--r-- 1 miro miro 11 Apr 13 20:53 /mnt/sde1/distfiles/timestamp.mirmon
-rw-r--r-- 1 miro miro 90336343 Mar 25 16:52 distfiles/VirtualBox-4.3.10.tar.bz2
-rw-r--r-- 1 miro miro 90333712 Mar 26 20:23 /mnt/sde1/distfiles/VirtualBox-4.3.10.tar.bz2
mybox somewhere #
[code]

Just in case, so we know what we are talking about (in Larry's "possession"'s
case):

[/code]
mybox somewhere # for i in `cat  distfiles_CHEK_overwrite.txt|grep VirtualB` ; \
   do sha256sum distfiles/$i /mnt/sde1/distfiles/$i ; done ;
8152fcc959565fee63855dffb9731a1585563f01b4756def0a644de1223af37e  distfiles/VirtualBox-4.3.10.tar.bz2
739835aee3274a663b23eeb748bd0430e8a5d8ba2f4d0eae5dc47ff2c485e23b  /mnt/sde1/distfiles/VirtualBox-4.3.10.tar.bz2
mybox somewhere #


I hope this will be looked into by people who know and can do more to make
these matters better. Because like _this_ is not good.

Without devoting more research into the matter, how is it distinguished the
previous and the new Google archives in the mirror, when they hold the same
name. Just as a sidenote. I need to go back to what I started this topic for,
can't spend much time on GeoIP and comrades.

I haven't, otherwise, with my offline install reached much further. Such as,
I sill need to read (again) those man pages, and it really takes me time
discovering these and those mechanisms, so I can eventually figure out how to
configure my system to be certain it installs safely without intrusion and
other non-free problems...

Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Tue Apr 15, 2014 6:59 am    Post subject: Reply with quote

Further progress and further issues.
These are a few lines, with comments and questions along, from my make.conf
Code:

# These lines should get me some logs for some period saved for troubleshooting:
# However, not solved yet how to change the -mtime +7 (7+ days old are deleted)
# to longer period. I want to keep all logs for longer (see the emerge --info)
# Ah, I think I figured out that line from man make.conf, see the last line of
# this paragraph with PORT_LOGDIR_CLEAN command, it's from man make.conf
PORTDIR="/usr/portage"
PORT_LOGDIR="/var/log/portage_logs"
PORTAGE_ELOG_CLASSES="info warn error log"
PORTAGE_ELOG_SYSTEM="save"
PORT_LOGDIR_CLEAN="find "${PORT_LOGDIR}" -type f ! -name "summary.log*" -mtime +90 -delete"
# yes 90 days, because text is cheap, and it's expensive when it's missing

DISTDIR="${PORTDIR}/distfiles"
PKGDIR="${PORTDIR}/packages"

PORTAGE_GPG_DIR="/etc/portage/gpg"
# allegedly disables 'emerge --sync', not clear to me, but not a hindrance:
SYNC=""
FEATURES="webrsync-gpg candy strict"
EMERGE_DEFAULT_OPTS="--keep-going --with-bdeps=y --autounmask-write --ask --verbose"

# grub wouldn't boot unless properly installed, for my PC this is needed:
GRUB_PLATFORMS="pc multiboot"

# For regular no mirror URIs fetching (lots of ebuild contain RESTRICT="mirror"
# and so those sources are not available from official mirrors), that will be a
# problem to solve how to safely get those, sieve off the potential
# intruder/attacker, check the system for freedom from all those and clone it
# as necessary
# But I'm inconsistent here. I don't get it... This neither is needed, since I
# now, upon update, have all what official mirrors have in my local mirror...
# No! I'll uncomment it. Not needed.
# GENTOO_MIRRORS="[some regular mirrors here]"
# because the packages with the RESTRICT="mirror" are anyway gotten from the
# devs' own websites or their sponsors', no variable needed in here for that.

# Instead of the usual GENTOO_MIRRORS line from the handbook, we go for the local
# mirror, but the local mirror is not written here, but in:
# /etc/portage/mirrors with a line such as:
# local http://192.168.N.N/gentoo/distfiles
# where N.N. will be according to where I put the mirror on my SOHO.


I have one more issue left to solve. I want to have it logged where I fetched what
package from. Because there will be offending packages. The most interesting is
when you figure out where they're from. Of course they're not usually
maliciously put there by the owners...

But the default for emerge-fetch.log is reflected by what we have in the man
emerge:
Code:

FILES
...[cnip]...
/var/log/emerge-fetch.log
   Contains a log of all the fetches in the previous emerge invocation

And that means once you fire up a new emerge invocation, the previous logs were
fed for dinner to your cat. No more there!

I don't like that. How do I change this? That is the sole relevant mention in
connection to the evasive fetch logs in man (emerge|make.conf|portage), if I
wanted to keep those logs by virtue of the portage configuration, is it at all
possible, or do I need to learn Python and make my own portage overlay, next
year when I'm done studying it, and change some obscure portage program from
/usr/lib/portage to have those logs, which is great, only I wouldn't be able
to do it in even more than that long time, probably, not with my free time
available...

Or do I need to combine a shell script. Or is there some way to run some script
exactly when emerge is called... I mean, is there a way here elegantly, with
the tools of the system. I sure could copy the previous emerge-fetch log from
the command line before running emerge. But that's a nuissance having to do
that.

Because anyway, I want those logs. Where I got what from...

I won't build my master system too often. Once a month has been less then my
average in between the emerge resyncing and compiling... But I want to know
what it is built from, as best I can.

I just looked up, and:
Code:

grep -rIl 'emerge-fetch' /usr/lib/portage/
/usr/lib/portage/pym/_emerge/Scheduler.py
/usr/lib/portage/pym/_emerge/EbuildBuild.py
/usr/lib/portage/pym/_emerge/Binpkg.py

and the Scheduler.py that calls the logging module at start, all is way beyond
my grasp.

M.R.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Tue Apr 15, 2014 10:45 am    Post subject: Reply with quote

I ran Clamav antivirus on my local mirror.

I deem it is useful for the community to point at issues (or non-issues that
appear as such) that Clamav screamed some about. Clamav FAQ should be a stop
(for not-much-initiated in the viri business like me).

Again, I can't dwell in here, I am a man in my early old age, and work slowly,
and my systems are long, long overdue to have been updated.

But, that is a long list of archives that potentially, or not at all if we hold
to the millions of users and developers who already used and viewed the code (a
rough paraphrase of:
NSA SELinux Support???
https://forums.gentoo.org/viewtopic-t-984066.html#7500374
((But I am not on that agency's topic here. Sick from the blowback.))
and found it impeccable):

...But that is too long a list of archives that Clamav potentially has some
problems with, and don't fit on the forums (some 270K). So:

http://www.croatiafidelis.hr/gnu/gentoo/clamscan_on_my-local-Gentoo-mirror_140414_16.txt.gz
http://www.croatiafidelis.hr/gnu/gentoo/clamscan_on_my-local-Gentoo-mirror_140414_16.txt.sig

As I said, I'm not dwelling here either. We, the community, others now, not me,
I am about done with the freetime for that, would need to do more.

Thanks to anyone who kindly considers my efforts.

Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr

P.S.
It's a rather long file,
Code:

 cat  clamscan_on_my-local-Gentoo-mirror_140414_16.txt | wc -l
3157

and subtract only 20 lines for the head and 10 lines for the summary in the tail,
so 3127 lines of Clamav's complaints in the sense, let's give some lines, just so
the DuckDuckGo can find it (if some more of free speech persist around):

grep gnome clamscan_on_my-local-Gentoo-mirror_140414_16.txt | head -5
Code:

/my-archive/distfiles/gnome-applets-2.16.2.tar.bz2: Heuristics.Structured.SSN FOUND
/my-archive/distfiles/gnome-applets-2.18.0.tar.bz2: Heuristics.Structured.SSN FOUND
/my-archive/distfiles/gnome-applets-2.20.0.tar.bz2: Heuristics.Structured.SSN FOUND
/my-archive/distfiles/gnome-applets-2.20.1.tar.bz2: Heuristics.Structured.SSN FOUND
/my-archive/distfiles/gnome-chess-3.10.2.tar.xz: Heuristics.Structured.CreditCardNumber FOUND


grep kde clamscan_on_my-local-Gentoo-mirror_140414_16.txt | head -5
Code:

/my-archive/distfiles/kdeaddons-3.5.0.tar.bz2: Heuristics.Structured.CreditCardNumber FOUND
/my-archive/distfiles/kdeaddons-3.5.10.tar.bz2: Heuristics.Structured.CreditCardNumber FOUND
/my-archive/distfiles/kdeaddons-3.5.4.tar.bz2: Heuristics.Structured.CreditCardNumber FOUND
/my-archive/distfiles/kdeaddons-3.5.5.tar.bz2: Heuristics.Structured.CreditCardNumber FOUND
/my-archive/distfiles/kdeaddons-3.5.6.tar.bz2: Heuristics.Structured.CreditCardNumber FOUND


But there are whole lots of other PUA.Win32.Packer, PUA.Script, PUA.HTML, and other stuff.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Tue Apr 15, 2014 12:09 pm    Post subject: Reply with quote

miroR wrote:
...
I have one more issue left to solve. I want to have it logged where I fetched what
package from. Because there will be offending packages. The most interesting is
when you figure out where they're from. Of course they're not usually
maliciously put there by the owners...

But the default for emerge-fetch.log is reflected by what we have in the man
emerge:
Code:

FILES
...[snip]...
/var/log/emerge-fetch.log
   Contains a log of all the fetches in the previous emerge invocation

And that means once you fire up a new emerge invocation, the previous logs were
fed for dinner to your cat. No more there!
...[snip]...
Because anyway, I want those logs. Where I got what from...
...[snip]...

I try and remember newbier beginners than me, so I'll just repropose here,
if anyone is trying to solve similar issues like (some of) these that I have
been solving here, that there is the hard-to-read-but-indispensable:

Code:
/usr/share/portage/config/make.conf.example


that upon a reread I figured out the solution to the emerge-fetch.log-fed-to-
your-cat issue.
The fetch command, not the new portage source overlay, is the solution for me
(of course I am joking against my own self, I am not Daniel Robbins, who gave
to the world this great portage architecture, for which he really deserves
credit. Sadly, things went as they went...)
The default is (you can see it if you issue 'emerge --info' is:
Code:
FETCHCOMMAND="wget -t 3 -T 60 --passive-ftp -O "${DISTDIR}/${FILE}" "${URI}""

All I will do, is I'll simply change it to, but in the make.conf there's
backslash-escaping to do:
Code:
FETCHCOMMAND="wget -t 3 -T 60 --passive-ftp -O \"\${DISTDIR}/\${FILE}\" \"\${URI}\" -a \"\${PORT_LOGDIR}/wget_fetches.log\""

man wget has that append-logging option. Will report if that doesn't work. Won't bother if it does.
M.R.


Last edited by miroR on Mon Apr 21, 2014 2:25 am; edited 1 time in total
Back to top
View user's profile Send private message
krinn
Advocate
Advocate


Joined: 02 May 2003
Posts: 4595

PostPosted: Tue Apr 15, 2014 12:50 pm    Post subject: Reply with quote

I'm sorry i didn't read it.
So that post would be useless, but in fact, i honestly don't think it's a useless one.

Your thread should be selected for the "The thread nobody would like to read" award...
Really, try open another thread if you need help, but WOW !!! Make it smaller !!! And i mean really smaller.
You would get more help with a small question on a clear define goal. You even quote yourself in it. It's impressive, but too much impressive for anyone to actually read it.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Tue Apr 15, 2014 1:55 pm    Post subject: Reply with quote

Hi, krinn.
krinn wrote:

...[snip]...
Make it smaller !!! And i mean really smaller.

Gladly if I knew. But some of the things, most of the things actually, I figured out as I went.
krinn wrote:

You would get more help with a small question on a clear define goal.

I believe the goal I have clearly defined. The method(s) I am figuring out and very painstakingly. Really can't do better.
krinn wrote:

You even quote yourself in it. It's impressive, but too much impressive for anyone to actually read it.

Yes I do because the FETCHCOMMAND line, a post or two previous to this, is the solution for what I really needed, and which I'm happy if I finally solved that knot. Also, maybe other users could use that command too.

Thanks for reading it, however doesn't help much,
M.R.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Wed Apr 16, 2014 1:50 am    Post subject: Reply with quote

Here tiny small hours in Europe.
But how could I have gone to sleep before rebooting into my completely offline installed system?

Oh no, I had to do it.

The local mirror works like a charm.

Basically I did figure it all out (minor corrections, but minor, here and there, but I'\m too tired now to systematically show which of them), in the last couple of posts.

I'm dumping the partitions now, it'll be a breeze (what's restoring 70G onto another HDD with exactly same partition table and size, on another same MBO but a breeze in comparison to all this research in this thread?).

Of course, I am on hardened-sources (as of course I used stage3 hardened).

Hardened sources of course, of freaking course in my case mean Grsecurity/Pax!

But one thing is missing, I think with the new LiveCD (but haven't tried it, this time I went with the SysrescCD -- www.SysrescCD.org just in case, really good!), but I know was missing in the LiveCD of 2014-02-27, and that is the gnupg is missing, I mean can't do emerge-webrsync really at all, before either emerge --sync to get also gnupg, or manually copying the gnupg packages via other means...

For a really safe system, manually getting the packages needed is what I did. And emerge-webrsync is incredibly much better than plain syncing.

Also, my assumptions of installing portage snapshot by hand, without emerge-webrsync, what I probably suggested can be done maybe two weeks ago (meaning: doing the emerge-webrsync's job of unpacking only, because the portage snapshots aloo can be gotten manually), that manual unpacking which I suggested in this thread (can go search, really tired), was completely correct. Just don't forget the p switch, as when unpacking stage3.

I'm also going with the:
ACCEPT_LICENSE="-* @FREE"

And this is basically the biggest hurdles are past now.

I reached up to Finalizing the installation in the handbook.

Ah, not to forget, the line for the FETCHCOMMAND for some reason didn't recognize the "$PORT_LOGDIR" variable that I gave it, but it is a minor issue, I just used simply /var/log instead, and wget is logging all.

There are bound to be other things to solve.

Will update, and will check if anyone is using this way and needs help. (Can't promise to be able, too many other potential problems unrelated to here, in my life, but will try. But patience is needed if anyone will wait for my reply here, I work slowly, am somewhat old)

Thank you!

Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Fri Apr 18, 2014 6:53 pm    Post subject: Reply with quote

I'm having problems installing, actually starting X after installing it, here:
Installing X; but X can't see what's wrong, only won't start
https://forums.gentoo.org/viewtopic-t-988956.html
Back to top
View user's profile Send private message
vaxbrat
l33t
l33t


Joined: 05 Oct 2005
Posts: 620
Location: DC Burbs

PostPosted: Sat Apr 19, 2014 4:08 am    Post subject: tailoring /distfiles? Reply with quote

On a somewhat related requirement, I would be interested in something that looked at your world list and pulled distfiles for all ebuild versions that were in portage. This would be for a scenario where the target systems are airgapped, and it would be useful to pull only incremental updates that are brought over via sneakernet. The btrfs snapshotting would help in figuring out the incremental parts.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Sat Apr 19, 2014 3:42 pm    Post subject: Re: tailoring /distfiles? Reply with quote

vaxbrat wrote:
On a somewhat related requirement, I would be interested in something that looked at your world list and pulled distfiles for all ebuild versions that were in portage. This would be for a scenario where the target systems are airgapped, and it would be useful to pull only incremental updates that are brought over via sneakernet. The btrfs snapshotting would help in figuring out the incremental parts.

Your suggestions are now being considered. Meaning:" ...[snip]... airgapped...[snip]... sneakernet...[snip]... " and possibly other things I need to get myself more familiarized with, through ddg-going (I don't google at all, down with the surveillors).
Thanks! Pls. allow some time.
On another note, I just solved:
Installing X; but X can't see what's wrong, only won't start
where admins who write the handbook, could look up my suggestion here:
https://forums.gentoo.org/viewtopic-p-7538416.html#7538416

M.R.
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Sat Apr 19, 2014 6:04 pm    Post subject: Re: tailoring /distfiles? Reply with quote

I studied some of (it's huge...) this good page:
https://www.schneier.com/blog/archives/2013/10/air_gaps.html
vaxbrat wrote:
This would be for a scenario where the target systems are airgapped,

So, people moving against surveillance, and it's growing! Good to see. And I see lots of stuff that I came to in my own somewhat flawed but hardworking and honest researches, such as can be seen on forums.grsecurity.net , if one searches for Miroslav Rovis, there is a lot there. Schneier is worth of respect, and many people there!
Yes, actually this topic that I started is about airgapping. Yes.
vaxbrat wrote:
and it would be useful to pull only incremental updates that are brought over via sneakernet. The btrfs snapshotting would help in figuring out the incremental parts.

I believe that btrfs only adds so much more complexity. I'm pretty sure that it does.
No, I wouldn't add it into the methods to use, doesn't help in airgapping.
The fact that is makes finding what to update on the offline system faster, well the rsync is there for that, and rsync, the program started by Andrew Tridgell, an Australian (IIRC) of the Samba fame, is pretty solid and I really like things by shiny honest developers like him, than any Larry Oracle's stuff... Larry ruined Java, didn't he? Put Oracle onto once truly free MySQL... Not an insider, but doesn't smell good.

vaxbrat wrote:
On a somewhat related requirement, I would be interested in something that looked at your world list and pulled distfiles for all ebuild versions that were in portage.


I see your point, but believe me, I actually relax when I see compilations and movements of files, I mean, the brunt of the work of these methods that I'm trying to figure, and of this howto not yet made, is in the understanding of how to use what is already there, the emerge, portage, ebuild checking (that is only here so perfect, thanks Daniel Robbins and all the devs, no other GNU/Linux has that much of perfection as portage!), and stuff, the great Pytthon put to such grandiose work, that is what needs to be phathomed and use for airgapping Gentoo!

I have to go back towards, hopefully, finshing my installation, because I'm long overdue using, and not building, my Gentoo boxes!

Pls. Gentoo devs, keep it privacy-viable and feasibly surveillance-free!

Miroslav Rovis
Zagreb, Croatia,
www.CroatiaFidelis.hr


Last edited by miroR on Mon Apr 21, 2014 2:26 am; edited 1 time in total
Back to top
View user's profile Send private message
miroR
Guru
Guru


Joined: 05 Mar 2008
Posts: 482

PostPosted: Sun Apr 20, 2014 7:33 pm    Post subject: Reply with quote

I'm now grappling with how to install LXDE without frills:
LXDE replacement question [SOLVED]
https://forums.gentoo.org/viewtopic-p-7538796.html#7538754
What is most important there, is I got rid of dbus consolekit and policykit flags, and I will add here that another flag that can be seen in my emerge, which I won't clobber with here, change being too little yet for posting, here:
( Installing X; but X ... freezes [SOLVED] )
https://forums.gentoo.org/viewtopic-p-7538746.html#7537924
is also I added
-introspection
into the USE bunch. Why? adds "support for GObject based introspection" (do grep introspection /usr/portage/profiles/use.desc), and it's default for lxde packages, not in mine anymore. Good when you can disable things. Problems if things not really needed are there, and you can't go without them...

I'm also trying to use AIDE, I have already initialized its database, and what I would like to do, is use it continually from now, such as first thing learning to update it once I install lxde in the way yet to devise.

I'd like to actually use AIDE in the way described in the Handbook:
http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part3_chap6__chap3_sect2

It'll possibly take me long to correctly use those pieces of advice, so it's really not like I can write brief and exact (simple) enquiry as krinn suggested ia couple of post age, no. It's I just can't be more precise, for flat lack of clear grasp on the matters.

But the air-gap Gentoo install has to be feasible, and for non-expert users like me.

Miroslav Rovis
www.CroatiaFidelis.hr
Happy Easter to everybody!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum