The newest, of today, is:
< this same topic >
https://forums.gentoo.org/viewtopic-t-1 ... ml#7643508
where I think I found an issue, maybe even a bug, and esp. there is a simple and easy to understand advice for surveillance-aware users.
---
Noooo! But I'm so glad I found it sooner than somebody rubbing my nose with it!
http://dev.gentoo.org/distfile-mirroring/whitelists.xml
It's there, the:
openssh-6.6.1p1-hpnssh14v5.diff.xz
---
But still! It can't be a right package with all the wrong HASHES... So... What do I do?... I most certainly can't install that package!
(This intro obviously will be sorted as I find out the solution, or am given advice.)
EDIT END
---
EDIT START 2014-10-14
While I can't separate the broken pipe syslog-ng issue from Mirror verification without a rewrite, I have posted separately on that issue here:
Syslog-ng from Delay in Logging to Broken Pipe and no Loggin
https://forums.gentoo.org/viewtopic-t-1001994.html
and will keep this topic for only Mirror Verification
{EDIT START 2014-10-25 and the entire reason for me going on this rampage here was for the issue that was solved as I explained there... But who could have known? Still, I won't give up on verification of the mirror. It should be possible to run a relatively simple program and know that all the files in your mirror are Gentoo Engineeriing Team verified... However, not at that level myself, and may never reach that level, to be able to do something of that kind myself....
EDIR END}
EDIT END
---
EDIT 2014-10-12 changing the title to:
Broken Pipe on Air-Gapped (and Verifying Your Mirror)
---
EDIT 2014-10-11 START
Just figured out some things, and so part of the suspicions here, about portage snapshots not being there are likely completely wrong. They are just not out in the open for non-experts like me, but I'm figuring out...
Bear this in mind:
(this same topic)
https://forums.gentoo.org/viewtopic-t-1 ... ml#7631234
and sorry to have bothered the releng with this.
EDIT END
---
title:
Broken Pipe on Air-Gapped (and Portage Snapshots Unavailable on Mirrors Wordlwide)
---
To be able to follow what we have here, a newbie (I'm always about getting new people into true *niix Gentoo-no-poetteringware-no-spyware), a newbie needs to familiarize with:
Air-Gapped Gentoo Install, Tentative
https://forums.gentoo.org/viewtopic-t-987268.html
Woke up this morning after I fell asleep while a slow bash script was running, and saw this:
Code: Select all
# tailf messages &
[1] 8966
...[ 7 lines snipped ]...
Oct 10 03:16:09 mybox kernel: grsec: chdir to /usr/portage/app-i18n/jfbterm by /usr/bin/updatedb[updatedb:10093] uid/euid:0/0 gid/egid:0/0, parent /etc/cron.daily/mlocate[mlocate:10085] uid/euid:0/0 gid/egid:0/0
Oct 10 03:16:09 mybox kernel: grsec: chdir to /usr/portage/app-i18n by /usr/bin/updatedb[updatedb:10093] uid/euid:0/0 gid/egid:0/0, parent /etc/cron.daily/mlocate[mlocate:10085] uid/euid:0/0 gid/egid:0/0
Oct 10 03:15:57 mybox syslog-ng[2097]: I/O error occurred while reading; fd='9', error='Broken pipe (32)'
# date
Fri 10 Oct 10:32:48 CEST 2014
# ls -ltr messages
-rw------- 1 root root 61543257 2014-10-10 03:15 messages
#
And, as the issued "date" command of my typing tells, the tailf is not producing any new lines from the messages. And the messages is not written to at all.
LAST EDIT BEFORE POSTING START (upon proofreading):
Yes, that's still almost the entire lot of it in all of these ten (10) hours:
Code: Select all
# tail -10 messages
...[ 8 lines snipped ]...
Oct 10 03:15:57 mybox syslog-ng[2097]: I/O error occurred while reading; fd='9', error='Broken pipe (32)'
Oct 10 11:49:15 mybox sudo: ukrainian : TTY=pts/4 ; PWD=/home/ukrainian ; USER=root ; COMMAND=/bin/bash
#
Code: Select all
# tail -1 messages
Oct 10 03:15:57 mybox syslog-ng[2097]: I/O error occurred while reading; fd='9', error='Broken pipe (32)'
#
Also the "/usr/bin/updatedb[updatedb ..." lines. You are spared them and see
just those two. But:
Code: Select all
# tail -9000 messages | grep 'updatedb' | wc -l
8999
# tail -10000 messages | grep 'updatedb' | wc -l
9999
# tail -100000 messages | grep 'updatedb' | wc -l
99999
# tail -1000000 messages | grep 'updatedb' | wc -l
234924
# tail -234930 messages | grep 'updatedb' | wc -l
234920
#
Code: Select all
# tail -234930 messages | grep 'updatedb' | head
Oct 10 03:12:59 mybox kernel: grsec: exec of /usr/bin/updatedb (/usr/bin/updatedb -f ) by /usr/bin/updatedb[mlocate:10093] uid/euid:0/0 gid/egid:0/0, parent /etc/cron.daily/mlocate[mlocate:10085] uid/euid:0/0 gid/egid:0/0
...[ 8 lines snipped ]...
Oct 10 03:12:59 mybox kernel: grsec: chdir to /Cmn/BAKB1016_dd/dd_B0420/etc_at8-g200/.java by /usr/bin/updatedb[updatedb:10093] uid/euid:0/0 gid/egid:0/0, parent /etc/cron.daily/mlocate[mlocate:10085] uid/euid:0/0 gid/egid:0/0
grep: write error
tail: error writing ‘standard output’: Broken pipe
#
Code: Select all
tail: error writing ‘standard output’: Broken pipe
Also, this I think is the first time that I see that stinking rummaging through my machine without me initializing it. Namely I didn't set this looking up through all the files in my box in any way.
I saw it previously, it just rummages through all and any of whatever that is there, or that I mount, or uncompress somewhere, anything whatsoever. It's the kind of rummaging through that I remember first seeing when I tried Red Hat really long time ago, and later on with SuSE all the time, and to some extent recently on Debian, but never, never, never yet on Gentoo.
Exampli gratia, I uncompressed a few of my old portage snapshots, and this freaking not-initialized-by-me-in-any-way updatedb (the snooping that never yet existed in any implementation of any of my Gentoos) went rummaging through all of them:
Code: Select all
$ ls -ABRgod portage-2014????/
drwxr-xr-x 169 4096 2014-05-28 02:31 portage-20140527/
drwxr-xr-x 169 4096 2014-05-29 02:31 portage-20140528/
drwxr-xr-x 169 4096 2014-05-30 02:31 portage-20140529/
drwxr-xr-x 169 4096 2014-07-12 02:31 portage-20140711/
drwxr-xr-x 169 4096 2014-09-30 02:31 portage-20140929/
drwxr-xr-x 169 4096 2014-10-01 02:31 portage-20140930/
drwxr-xr-x 169 4096 2014-10-02 02:31 portage-20141001/
drwxr-xr-x 169 4096 2014-10-03 02:31 portage-20141002/
drwxr-xr-x 169 4096 2014-10-04 02:31 portage-20141003/
drwxr-xr-x 169 4096 2014-10-05 02:31 portage-20141004/
drwxr-xr-x 169 4096 2014-10-06 02:31 portage-20141005/
drwxr-xr-x 169 4096 2014-10-07 02:31 portage-20141006/
$
Code: Select all
$ du -sh portage-2014????/
826M portage-20140527/
827M portage-20140528/
829M portage-20140529/
835M portage-20140711/
841M portage-20140929/
841M portage-20140930/
841M portage-20141001/
841M portage-20141002/
841M portage-20141003/
841M portage-20141004/
842M portage-20141005/
842M portage-20141006/
$
While I have no idea yet why the errors in my /var/log/messages above, I know I can tell you that showing you this list of uncompressed portage snaphots does makes some sense, and you'll easily grasp why, next.
Late on 2014-10-06 (or early on 2014-10-07) I rsynced my private mirror. And the sole portage snapshots that I was able to download from, I tried a few mirros' /snapshots/ directories, are the ones shown above, portage-20140929.tar.xz{,.gpgsig,.md5sum,.umd5sum} through portage-20141006.tar.xz{,.gpgsig,.md5sum,.umd5sum} (excluding the old ones before 2014-09).
In other words, if anyone in the world now tried to get the verified portage snapshots other then (it's been days that I work on this, so the last 8 snapshots are different now: portage-20141003 through portage-20141010), they wouldn't be able to.
A month's worth of such great package verifiers that as a method is hard to find so splendid in any FOSS Linux/*nix is around 2GB. Is that an amount worth sparing?
Removing them like this, so very very soon, is like saying, oh well, I can sign my statements, but if you are not around every single freaking week, to ask for them, well then, I don't want you to be able to read my signed statements. Why that attitude?
I will try and ask the releng about this, and who knows, maybe I get this term prolonged for air-gappers like me to at least one month (three months would be really a decent minimum, but this one week is a shame! so anything even a bit longer would be an improvement)... Who knows maybe they'll lend an ear...
But the above is just an impediment to my problem, and not the problem that I am trying to describe and solve.
And to explain the problem to you, id est to eliminate a few possible causes, I need to explain the current Gentoo instance on which it occurs.
In the first place, just as I am a backup-dependent air-gapper and so I had a few previous portage snapshots lying around which the Gentoo devs are not letting its own world of users have for some unknown and obviously puzzling reason, and so I am amongst the tiny few of Gentoo users that could ever even do the following, in the world!...
In the first place, right after my problem showed up upon my research when I had strange events described on:
grsec: halting the system due to suspicious kernel crash
https://forums.grsecurity.net/viewtopic ... =15#p14456
and I mean the syslog-ng complaining about the "Broken pipe" and then not logging at all. At all! That sure is _the_ problem...
In the first place, right after my problem showed up, I expected I had to go pretty deep to solve it.
Having just sufficient knowledge to comfortably discuss most of these issues, I decided I needed to first go and see if the issue would go away if I simply just recompiled the system back from when I had no such issue.
And I actually went four (4) months back.
What I mean is, I restored the system partitions to the state they were four months ago, to the bit, yes: to every single bit (
for newbies possibly reading this, some explanation how I did it is here:
Postfix smtp-tls-wrapper, Bkp/Cloning Mthd, A Zerk Provider
https://forums.gentoo.org/viewtopic-t-999436.html
)
, from the backup that I took back then.
Then I updated the system in ways that is, by some unknown and puzzling design and similar sparingliness of 2GB per/month of not-so-terribly used snapshots
( most of people generally don't care about security, proof being Linus' words how security is not the most important thing [1] seemingly not bringing much reaction from the FOSS Linux crowd:
QA_with_Linus_Torvalds
http://www.youtube.com/watch?v=2mIPPKReeGg
(duration: 1:11:00)
to hear the kind rant of his against real security, go to 1:05:30, six minutes
before end
My take on it you can read:
Why is Gentoo not switching to systemd?
https://forums.gentoo.org/viewtopic-t-9 ... ml#7624044
)...
[Then I updated the system] in ways that is not available to almost anyone in the world, and that is, by using portage snapshots from 2014-06, then 2014-07, 2014-08 and 2014-09.
I want to point out here, that I would have needed the previous signed portage snapshots for very exact and very sensible reason, but I need to expand on it in my own next separate post, to keep the presentation a little less complex. Just, it is about verifying the local mirror.
The system was offline all the time since the restore-and-reemerge. And I surely didn't connect the IPTV recording clone of its to it which caused the kernel panic issue, unresolved yet, that I posted about on the Grsecurity Forums topic in the link given above.
It actually is not and does not connect to anything that is in any way online. This system is, as far as a user can achieve it, air-gapped.
( And, just to tell this much about verifying of what I went to install, and whether there could have been extraneous causes to this problem introduced from unverified packages: there weren't any unverified packages installed, because I checked every single download, and from pretty carefully maintained local mirror!, before emerging any --about that more in the next post that I am yet to write. )
And, upon the update to 2014-09-xx, let me see precisely:
Code: Select all
# ls -ABRgo /usr/portage/distfiles/portage-20140910.tar.xz*
-rw-r--r-- 1 61564992 2014-09-11 02:45 /usr/portage/distfiles/portage-20140910.tar.xz
-rw-r--r-- 1 819 2014-09-11 02:55 /usr/portage/distfiles/portage-20140910.tar.xz.gpgsig
-rw-r--r-- 1 58 2014-09-11 02:55 /usr/portage/distfiles/portage-20140910.tar.xz.md5sum
-rw-r--r-- 1 55 2014-09-11 02:55 /usr/portage/distfiles/portage-20140910.tar.xz.umd5sum
#
So, while I will try and see if the problem goes away if I try and update the system with, say portage-20141006 snapshot, I still wish to post this, because with this post, I want to also kindly ask the Gentoo Release Engineering Team (the releng) to prolong this term for security-aware users, such as air-gappers like me, to at least one month. Three months of keeping the portage snapshots on the mirrors would be really a decent minimum, but this one week is a shame! And so anything even a bit longer would be an improvement...
Pls., dear Gentoo releng, lend an ear to this need of I believe many other users as well!
Because, it is, this making unavailable of these great package verifiers that signed portage snaphots are, after just those few days, it is like saying:
Why that attitude?Oh well, I can sign my statements, but if you are not around every single freaking week, to ask for them, well then, then I don't want you to be able to read my signed statements.
I'm ill these days, so pls., while I will try and respond quickly, allow longer time for me to reply if you post any followups.
Regarding the post in which I intend to explain how I'm trying to verify my local (private) mirror and new downloads from it before installation, allow time for that too. On top of having issues with my health, I really work slowly anyway
Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr
[1] What Linus says there could be paraphrased: "Security is not the most important thing in the kernel, performance is more important." To which the right reply would be: "Fine. Give me the keys to your apartment and feel free running around the town, "stupid"! [2]". So Linus is wrong! And leaving out the spyware SELinux imposed on all the newbies of FOSS Linux in the world which someone needs to tell them what it is, and less and less information there is in this censored world about its true nature... Leaving the spyware SELinux out, the most important thing for a user with at least some brains, is:
Apartment is of course allegory for my computer.I want my apartment locked for anyone but who I intend to allow in it, when I'm having my time downtime(, "stupid"!)...
[2] "stupid" is TM by Linus Torvalds, because that's a reference to that guy's attitude, I don't otherwise like calling people names in public like he does. My telling him that he is a lier, as I wrote in the link to my post in "Why is Gentoo not switching to systemd?" is not calling him names, it's truth, and it's based on logical reasoning derived from two expert arguments (not mine) that I have not yet seen rejected, and I doubt that anyone can, really reject.
I really wish somebody should at least try and reject those two arguments, instead of this almost silence on them.
Where are you FOSS people from other countries, from Russia, from China, from Brasil, from Africa, from anywhere else? Do none of you at this day and age speak English fluent enough to tell your thoughts? Or what is the reason? FOSS Linux, and Gentoo, belongs to us all, not to any culture in particular, but to all nations and all cultures, because we all contributed to FOSS Linux/*nix and it's ours...

